Spatial Computing

https://hypercritical.co/2024/01/30/spatial-computing

by ingve • 2 years ago

88 81 2 years ago

mcbishop• 2 years ago

I think the eventual "direct manipulation" is to manipulate directly with the mind... no physical movement required.

blkhp19• 2 years ago

I could see there being serious issues with handling "impulse control." Imagine you see an ad, have a brief desire to interact with it, but tell yourself a split second later that you shouldn't. The time delay of using our physical body to carry out actions (movement, speech) that originate as thoughts acts as a very necessary filter. Imagine a world, even isolated to a spacial computing environment, where this filter didn't exist.

yurishimo• 2 years ago

I think there are two minds to this.

One, with software, you could delay any action for a set amount of time as such to give your brain time to make a more concrete decision. Idk how long that delay needs to be, but I'm guessing it's less than 300ms.

On the other hand, if the hardware is responsive enough, you can simply decide to close the program/ad/whatever another split second after that fleeting thought.

The pessimist in me thinks that the first option is probably the best one given software optimization in the past two decades, but who knows.

eviks6• 2 years ago

Why wouldn't you simply add a filter if it's an issue?

trinsic22• 2 years ago

I don't know if the Visison Pro is like other 3d head sets, but I realized there is more of a cognitive load on the brain then just using a 2d screen. I think it has many applications, but I wouldn't want to be using it for long periods of time.

CoolestBeans• 2 years ago

You really put into words something I've experienced but couldn't articulate. Every time I've used a headset I've gotten not fatigued but just a general feeling of malaise and had to put it down and take a break. I would imagine with repetition you get used to it but yeah its definitely a factor.

Although I think part of the bleh feeling is just having to subconsciously adapt to the imperfections of a headset. For example, you can't just glance at anything and expect to see it, you have to shift your head for some things. Or like the headset itself moving ever so slightly on the face and your eye moving out of the way of the center of the lens. It gets exhausting and you don't know you're doing it unless you're actively paying attention.

I will say most of the advertising shows people interacting with 2D interfaces in 3D space so maybe this is how they're getting around it combined with just better build quality, better thought out straps, etc.

trinsic22• 2 years ago

Yea maybe you can get used to it. I have only used heat sets a few times.

One thing I noticed was that My brain had two visual maps competing against each other. Reality and the virtual world. I sometimes found myself feeling the sensations from the virtual world from time to time while I was not using the head sets. It went away with time, but it was interesting anyway.

threeseed2• 2 years ago

The whole point of the eye/finger mechanism is so that you can just pickup the headset at any time, watch a movie or play a game without needing to worry about finding and charging controllers.

For doing proper work the keyboard/trackpad combination looks to be ideal.

realharo• 2 years ago

For navigating menus, even the original Quest's hand tracking, where you used your fingers to point at things, is good enough.

crazygringo• 2 years ago

The Quest 2's hand tracking is so bad as to be literally unusable for me.

I just cannot get it to reliably recognize pinching. It's like only half the time, and sometimes I have to pinch like ten times in a row until success. Or the act of pinching also moves the hand off the item you're trying to click.

And then certain things your on-screen hands just can't reach. Super unclear why. But I think something to do with trying to map hands close to you, to an interface button visually one or two meters away.

jojobas• 2 years ago

12 years after Oculus Rift SDK VR is still a gimmick that only ~10% of the relatively few owners use weekly. Calling it "spatial computing" won't change it.

ildjarn• 2 years ago

The second screen stuff has commercial applications.

muunbo2• 2 years ago

One important aspect missing is tactical feedback. The mouse, keyboard, and touchscreen all provide that but Vision Pro doesn’t. We have evolved several nervous system sensors (touch, pressure, temperature) for this and it is just as “human” as spatial reasoning and hand manipulation of objects

meheleventyone2• 2 years ago

That combined with a noisy signals from inferred controls with eye and hand tracking will frustrate a lot of people. Our current interfaces are extremely precise in comparison.

I quickly narrowed down on VR experiences that provide a decent feeling of facsimile to real life being the most interesting and engaging. Beat Saber is a case in point here but for me it was combat flight sims using stick, throttle and pedals. Moving to gestures and eye tracking wrecks all of that.

I also don’t know why the original article calls using a mouse indirect as if we aren’t masters at that as well. As well as a spatially aware species we are also a tooling using species. We are experts at leveraging our spatial reasoning to manipulate tools in the world to do things we otherwise couldn’t. A mouse movement in the world translating to moving a cursor on screen is a perfect fit.

threeseed2• 2 years ago

There is tactical feedback on the Vision Pro for selecting, scrolling and moving.

I suspect that we are going to see most apps centre around this finger/thumb motion.

dvh2• 2 years ago

Do you and op mean tactile?

eviks6• 2 years ago

There is no tactile UI feedback in VPro, but then neither is it there in the mouse/keyboard. Otherwise when you touch your fingers against each other for some gesture, you feel them.

alan-hn• 2 years ago

Physical objects being physical things that you can touch is the feedback. Mass, inertia, and friction

LeoPanthera• 2 years ago

There's one huge advantage to the "indirect manipulation" of the Vision Pro - you aren't limited to the space around you that you can reach with your hands.

The Vision Pro allows you to manipulate things that are too far away to reach. I don't know if this will "feel" good or not, I won't know until I get one in two days, but it will either feel like a superpower - or be annoying.

Or possibly first one, then the other.

meheleventyone2• 2 years ago

This has been true since the earliest VR controllers of the current generation so there is plenty of evidence to check out right now!

vatys• 2 years ago

Virtually reaching for things far away might be interesting.

I've alway been on the fence about VR, even for games, and never found any experiences to be compelling after a few minutes of fun.

However, one thing that struck me about Half-Life Alyx was the "magic" feeling of the Gravity Gloves in the game. In the game you point and reach for something, flick back, and it flies into your hand. It's very satisfying and intuitive, and possibly my favorite aspect of the design of that game.

I'm not sure if this would translate well out of a game design and into "real world" mixed reality, but it's an interesting thought.

euroderf2• 2 years ago

IRL I can toss things in the corner of the room, or add them to a pile on the cabinet. My spatial memory will make a note. Soon I have teetering towers of klunk. This in accordance with Ringwald’s Law of Household Geometry.

So hey why not in VR/AR too ? Then I can easily levitate an entire pile to check thru it, like the way I can click on a stack on my Mac.

orangepanda• 2 years ago

Kids in 2034 will flick at an old-timey light switch from half a room away, and be confused why it doesnt work

yencabulator5• 2 years ago

Kids in 2034 will walk into a room and wonder why it's still dark, surely the everloving BigBroAI should have watched them do it and adjusted the lights already. Then they realize it's because their Internet connection failed, and really get afraid of the dark.

dragonelite2• 2 years ago

Did they renamed AR/VR to spatial computing?

blululu• 2 years ago

This has been a push for a while. It is pretty hard to explain the nebulous distinctions between AR, VR, MR and XR without coming across as a self-important pedant. Spatial computing is a much clearer umbrella term since it describes the core feature that is unique to this family of technologies.

fragmede• 2 years ago

I'll be the pedant then and say that how I understand it is that AR is just adding things, VR is when it replaces things entirely, MR is more than AR because there are digital and physical elements to interact with. XR, then, is an umbrella term that encompasses all of the above. Spacial computing vs XR as the phrase to use? Sure.

dragonelite2• 2 years ago

Aah this makes sense to put it under one umbrella of spatial computing.

mgaunard• 2 years ago

Touch interfaces are awful and a step backward.

vagrantJin• 2 years ago

Among the evil things my mind has cooked up, typing on touchscreen mobile devices is punishment.

dvh2• 2 years ago

For me VR for office use doesn't survive the "timeline reversal thought experiment".

Imagine that from the very beginning computers only had virtual reality headsets as the only interface. In the time of ENIAC it would be extremely huge grotesque steampunk helmet that you would put on but as the time goes by smaller and smaller headsets developed and eventually today we would arrive exactly where we are now with this Apple VR headset. And now, someone invents desktop PC monitor. It will be revolutionary. No longer you have to strap on headset and wear it for 8 hours when you're in the office. Instead of $4000 fragile gear you buy cheap $100 dollar flat panel with double the resolution. No motion sickness, no batteries, no cables around your neck. Everybody would switch and VR headsets would become niche product.

arijun• 2 years ago

That’s a bit silly of a thought experiment, because nothing survives it, since everything has its niche. If we had started with smartphones, computers coming along with their big screens and external input devices, people would be flooding to them. Even the timeline we have doesn’t survive—when smartphones came out it turned out people love having computing accessible from their pocket.

xanderlewis2• 2 years ago

That’s a bit silly of a thought experiment, because nothing survives it

I don’t think that’s true. There are developments that really do constitute genuine progress: for example, going from CRTs to LCDs. It’s hard to imagine a world populated by high-resolution, energy-efficient, colour-accurate flatscreens suddenly being replaced by the invention of a gigantic ugly box that performs worse on every available metric.

…and yet CRTs still exist as a ‘niche’ product. There’s a whole corner of the internet where people admire and collect them.

It may well be that ‘spatial computing’ or whatever you wish to call it is going to be a genuine step forward, but it’s also easy to imagine it might not be. And that’s not true for every technology.

vineyardmike• 2 years ago

For me VR for office use doesn't survive the "timeline reversal thought experiment".

I think the most damning "thought experiment" is "do you think you'd buy this?". I'm waiting for a VR headset for this. Less 'in office' and more 'WFH' but still same idea. I don't know if the Apple headset is the one (because my work/Apple won't let me connect my existing computer to it) but it looks to closely solve the problem on a hardware level.

I've screamed it from the mountains. I want a VR headset that acts as a monitor/tv and lets me turn any flat surface into a private desk. I travel a lot, and it sucks not having a monitor in a hotel. It sucks needing the extra room for a desk when I have tables already. Housing in my city is $1000+/sqft, so space for a desk or office is a meaningful expense for many. It sucks that having others in my house (kids, spouse) means I can't have a private, distraction free area without paying for a whole extra room (again, a premium).

Plenty of people besides me live in an n+1 bedroom house, where they have a whole extra room for a desk. Plenty more people live in urban environments where they can't afford extra private space. Think on the global scale, where people have more multi-generational homes, and live in smaller urban apartments. Sure they might not be able to afford the top-line apple headset today, but think longterm. Plenty more people are students, in academic halls and dorms and libraries, and would benefit from "on demand" distraction free, large monitors but don't have the private space for it.

josephg2• 2 years ago

Yeah, I watched a few videos about the apple headset and this was one of my takeaways as well. Would I buy this? - I would if it were a viable replacement for my computer monitor.

I have a 48" monitor at home. Its basically a TV on my desk. I absolutely adore having so much space while I'm programming. If the resolution, price, battery life, weight and sweatiness issues of virtual headsets were all addressed to the point that I could use a headset like that instead of a giant monitor, I'd totally buy one. Especially if it meant I didn't have to lug my laptop around as a result. After all - the apple headset has an M2 chip in it. There's no good technical reason to need a laptop as well.

The apple vision headset looks like its getting much closer to that dream. The displays and the software stack look much more appealing than the quest 2 I have. But they need to address the weight and battery life. And it needs to be a lot cheaper (or a lot more capable) to justify its price. I'll probably grab one in a few generations if they keep iterating on it.

cyanydeez• 2 years ago

ASUS Appears to be on this track.

https://edgeup.asus.com/2024/asus-glasses-intro/

sureglymop• 2 years ago

However, if we had glasses that looked like and were as light as regular sunglasses that had similar functionality but over wireless? That could be cool (and I suppose is the original google glass idea).

gautamb0• 2 years ago

Here's the thought experiment I'd run.

If you have a device that looks and feels like traditional glasses, but enables you to create and place an unlimited amount of virtual displays of any size and shape you desire, which can follow you around as you walk, why would you ever want to replace those with physical displays?

layer8• 2 years ago

Since I wore glasses for the first time as a kid, I never particularly liked wearing them, due to their tendency of fogging up, the way the frame impairs the field of view, and the weight on the nose and ears when wearing them for many hours.

Certainly, I would have some uses for such AR glasses, but I wouldn’t like to wear them all day unless they are literally (and completely unrealistically) invisible and weightless. They won’t replace displays.

What I’d really like is paper-like non-emissive, high-contrast, full-color-gamut, millimeter-thin, wireless multitouch displays I can hang everywhere. Those are similarly unrealistic though.

IshKebab2• 2 years ago

That sounds cool but also nothing like the AVP so it's not really relevant.

Cacti• 2 years ago

But it’s not a VR headset. It’s an AR headset, and it’s rapidly headed to convergence with VR headsets. After all, VR is in some sense a special case of AR.

VR headsets are just a stopgap until tech or price/power lowers enough to make AR more than a gimmick.

AR is the future, and everyone will have to have it the millisecond they start watching a demo of the Super Bowl being played out in your living room. Or pornography.

wolframhempel• 2 years ago

I'm a bit annoyed :-). We're building a platform (https://hivekit.io/) that lets you visualize, coordinate and automate operations in the physical world (on mines, construction sites, cities etc.).

When I first started talking and writing about it, I described what we did as spatial computing because - well - its computing stuff in a spatial sense. I even went further, suggesting that the patterns that emerge from entities operating in a physical space resemble the mechanics of other "thinking systems" like Neural Nets. (see https://hivekit.io/blog/building-ai-without-a-neural-network...)

I felt, the term spatial computing was super applicable for that - and I still feel that way. But with the release of the vision pro, we had to surrender the term to Apple and go with the less pompous "workforce orchestration" instead.

duped2• 2 years ago

Apple did not invent the term "spatial computing" to apply to XR (augmented/virtual/extended reality headsets). I've heard the term for at least the last ten years, personally. I believe Magic Leap first started using it. edit: wikipedia says it's been around since the 90s for VR

jwr• 2 years ago

Alas, our spatial computing journey seems to have taken many steps back. Our OSs treat icons on our desktops as theirs to do with as they please, rather than ours to keep where we want them. Window positions are forgotten, new windows are created pretty much at random anywhere the OS pleases. We can't arrange our files spatially. Instead, the OS talks to us through those dreaded "file open" dialogs, using abstractions of tables rather than spatial arrangements.

We can't even arrange icons on our iOS home screens as we please — we have to stick to a predefined grid with predefined sizes and use a terrible interface to move them around.

I do hope this will change in the future: our brains are wired for spatial memory and we should be making things easier for us by building software that is subservient to our needs, not the other way around.

jayd166• 2 years ago

It's not all bad. Windows finally fixed their window layout to monitor setup mappings. These VR headsets will try to keep things spatially anchored although the anchors still get lost fairly often.

vmfunction• 2 years ago

Apple had a file manager call x-space or something like that. Basically see your files in 3d space like game. It got killed by Steve of course at some point.

euroderf2• 2 years ago

When last I read about the Spatial Finder, years ago, it did not compute. Now suddenly it does. Chalk it up to Windows-induced brain damage.

Evolution: opposable thumbs => front panel loader => command line => GUI => (you are here) => spatial => true cyberpunk cyberspace.

Tired: "Two dimensions ought to be enough for anybody."

kbf• 2 years ago

It’s worth mentioning that there are accessibility options that let you control the pointer using your index finger or wrist instead of your eyes and it looks pretty nice. I do think that Apple chose eye tracking in large part to combat fatigue and not require external hardware for basic operation. For longer sessions where you’re constantly interacting with apps, a mouse and keyboard is the best option just like on Mac.

duped2• 2 years ago

What is so different about the AVP that sets it apart from the variety of headsets that have been on the market for years, besides the fact it's made by Apple?

YoshiRulz• 2 years ago

Passthrough resolution mainly. Also no controllers.

RamRodification• 2 years ago

The fact that it has an Apple user interface is a big one, I guess.

phailhaus• 2 years ago

Two things that set it apart:

1. R1 chip, which means that the spatial tracking basically never fails and is way more stable. Typical VR headsets will start lagging with the more CPU load they have, which causes disorientation and dizziness.

2. Eye tracking and gesture controls. There are cameras point at your eyes, down, and far to the sides. This means that no matter how you're sitting or where your arms are, it can see you "click" with your fingers. This level of seamlessness is necessary for the headset to feel effectively invisible. Typical VR headsets require you to deal with external controllers, which makes them more like consoles. Makes it more awkward to do simple things like check your phone or grab a drink.

I honestly don't think it's going to take off because nobody has demonstrated a "killer app", but the AVP is substantially different from other headsets in the space.

jayd166• 2 years ago

RE: 1. Do the pico headsets do this or something? Who doesn't have their spatial tracking process set as the highest priority?

Are you confusing dropped frames with spatial tracking loss? Apple hasn't invented a technology to never drop frames so I expect once we hop into fully immersive Unity apps we'll have familiar stuttery moments.

jayd166• 2 years ago

- 10x the parts budget. The Quest3 is good but the assumption is the AVP can be luxurious.

- It can leverage the Apple app store. While android headsets can sideload and run Android apps fine, none have the support of the Play Store.

- No controllers is a bold choice. Didn't work out for the hololens but should be better tech this time around.

IshKebab2• 2 years ago

* Pass through latency

* 6DOF tracking quality

* Ability to integrate with a Mac

* General high quality Apple software

* The hand and eye tracking features

I still don't think it will catch on. From all accounts it is less comfortable to wear than other headsets. Even with lighter headsets like the Quest there's literally no way 99% of people would choose wearing that for their normal work day over just using a normal computer. It's nowhere near comfortable enough.

People tolerate it for VR games because that's the only way you can play VR games. Apple seems to be targeting the AVP at stuff that you could just do on a laptop already.

Spatial Computing

Related Stories