VR optics and why IPD means too many things (2019)

https://tomforsyth1000.github.io/blog.wiki.html#%5B%5BVR%20optics%20and%20why%20IPD%20means%20too%20many%20things%5D%5D

by luu • 3 years ago

121 64 3 years ago

tomforsyth1000.github.io

Wow, what a fun and knowledge-filled article. It felt like reading Feynman.

Just casually browsing HN, this solves a problem that's been bugging me for months in my game: The world is scaled in feet/inches, while WebXR uses meters/cm to set ICD (virtual camera distance)! I think- based on my getting unusually nauseous.

I thought the solution was to scale the world, but there's more to it - and I think the solution is to reduce camera translation motion by the scale difference:

Specifically if you translate your head sideways by the distance of your IOD, your left eyeball should now be seeing what your right eyeball was seeing. If it doesn't, the brain worries that something bad has happened, and makes you feel bad about it.
Note that if you scale BOTH – you move the virtual cameras closer together, AND you reduce the in-game motion of the cameras due to head motion – everything is totally fine! The world just grew or shrank magically, but it all still works and you don't get nausea.

You may wonder why in hells a programmer would use imperial units instead of metric. It's to give the game a sense of being human-scale and pre-civilization. Why not just use metric and then convert everything to imperial for display purposes? Because then during debugging, units in logs constantly have to be converted to units shown, and the conversion isn't identical. It never bit me until VR!

teddyh4• 3 years ago

Imperial units are not “human scale” any more, or less, than metric: https://xkcd.com/526/

contrarian1234• 3 years ago

A 1 foot sandwich can be easily shared with 2, 3, 4 or 6 friends. And nobody has 5 friends anyway. There are no meter long sandwiches .. and if there were, you couldn't split it between 3 people. It's impractical. It's like if there were 100 degrees in a a circle.. You'd go nuts

Metric is for when you've made a mess of things and have completely given up on having nice round integer values

EDIT: This comment has been timed to coincide with when Americans wake up to defend me :) ~ USA! USA!

interestica• 3 years ago

I never understood why we never popularized a comparable metric measurement. A quarter-meter perhaps?

blululu• 3 years ago

You are citing a cartoon with a dick joke and a mom joke to say that metric is just as human oriented as imperial. Seriously: Imperial: 1 Foot ~ the length of some guy's foot. 1 Second - the duration of one heart beat. Metric: 1 Meter - some 1/10000 of 1/4 of an mis-measurement of the circumference of the Earth. 1 Second - stolen from the Imperial system cause 86 microdays did not easily divide by 10, 7, 24, 60, 30, 31 or 29.

taeric• 3 years ago

I mean, sorta? First, arbitrary numbers are arbitrary. But I can point to my foot and say that is about a foot in size. An inch is roughly one of my knuckles. A yard is roughly a step.

Even that comic you linked drives this home. None of them are referenced to humans. They are things we mostly know, sure. But they aren't based on a human. (Well, I suppose the mass ones have an amusing set of people listed.)

And this avoids the measures that, for my memory, just haven't made it to metric yet. Acre, Astrological Unit, Knot, etc.

IYasha• 3 years ago

"3 L - TWO-LITER BOTTLE"

Um, okay...

HPsquared2• 3 years ago

As computer specialists, we should advocate a return of the Imperial system using fractions of an inch for measurements. This is because it fits perfectly into the binary floating-point system. There are no more problems relating to 0.1 being an irrational number in binary. It's all broken down into 1/2, 1/4, 1/8, 1/16 etc which in binary is 0b0.1, 0b0.01, 0b0.001, 0b0.0001

h2odragon4• 3 years ago

How is it that there's any actual science left to be done here; when the "VR world" has been spending money in occasional waves since the 90s?

Shouldn't all this research have been done, and published, already? And re-verified, folded, spindled, mutilated, then recycled as firelighters. It's been 30 years and should mostly be adapting numbers optometrists know already.

teddyh4• 3 years ago

The first big VR wave was so early it was almost pre-Internet, so I assume most research was done in corporate silos, like Disney. Those silos are black holes from which no information can escape, so everybody has to re-invent everything.

h2odragon4• 3 years ago

1991; I spent a week manning a booth at a construction trade show. We were beside the Autodesk booth, where they had a big show of "VR walk through of the Denver Airport". Several times every day, all week, people would stagger out of the demo and yak into the trash can provided for the purpose... which was just behind us.

I think that really showed me the potential of the VR business and the spirit with which it would be pursued.

addaon• 3 years ago

1968 was significantly pre-internet [1], even beating out the first of ARPAnet by a few months.

[1] https://www.historyofinformation.com/detail.php?id=861

thebigwinning• 3 years ago

Since we have a lot of theoretical science, it's easy to imagine everything is solved in engineering. But when you dig into a new application you learn there are tons of gaps in knowledge. There is theory to draw on, but you need to combine and resynthesize to make it into something that works.

I have had experiences translating a professional model (similar to optometry prescriptions) and learn the model isn't described well enough to write in software.

jayd16• 3 years ago

Do you expect everything that existed in the 90s to be "complete" or just VR?

krisoft• 3 years ago

Good question. As a comparison we have been piling up dirt significantly longer than VR has been around and yet the physics of granular materials is still actively researched.

washadjeffmad2• 3 years ago

Translation into products requires a study of humans that most companies are ill-equipped to apply. For example, the Oculus/Meta Quest 2 headset only has 3 "IPD" settings, and the face plate was designed to fit a somewhat particular skeletal structure.

Notably, while it fits comfortably on my Western friends' faces, none of my East Asian friends or family, including any women, found it bearable (too tight on forehead, tilted downward, light leaking from around nose) or were able to focus it cleanly and assumed "VR was not for them".

However, a Japanese headset, often reviewed in English as being too small or uncomfortable, fit me perfectly, and the optics were clearer despite having lower technical specs. How illuminating!

I do wish that companies that designed wearables would either disclose the dimensions of their testers or provide what dimensions matter. I'm larger than 50% of the people on the planet, and I fall within a standard deviation of the mean in the United States. However, no product designed by a US tech company has ever fit me.

The real answer is probably the same as the Alf Landon election prediction by Literary Digest in 1936. Because they only sent out ballots to addresses of registered car owners and people listed in the relatively new telephone directory, the poll results skewed wealthy, urban, and conservative - class bias in action.

Are tech companies in the US making the same mistake?

reaperman• 3 years ago

Are tech companies in the US making the same mistake?

Yes, but slightly different, because they're missing less of their "addressable market" than the Landon prediction was. Their addressable market generally being people outside of Asia who have lots of disposable income. People in Asia tend to buy homegrown products which approach or exceed the quality, are easier to get, and cheaper. People with disposable income outside of Asia still trend toward the Westerner morphology. Still it's amazing with all the East Asians in big tech companies, that so many domestic products seem ill-designed to fit them.

Meanwhile tons of my furniture and other items that are designed and manufactured in Asia and then shipped to the USA seem to be shrinking and I don't fit in so many of them anymore. Not sure if that's just selling cheap things that have less material/shipping cost, or if it's matching the optimal sizes for East Asians.

thendrill2• 3 years ago

VR AR researcher here.

It is scary once you see what the actual VR research landscape looks like.....

There is a clear disconnect between users of vr and vr researchers.

I don't know even where to begin. Too much incompetence. Too much disinterest. Too much bullshit just to get grant money. And nobody has any clear idea of what VRs application should be.

So far even for me there are very few applications where vr gives an actual advantage for anything.

Take it situational awareness, overview, ux, visual quality etc. Most of the vr apps I have seen have no reason to be in vr.

Darkphibre2• 3 years ago

I hear ya! One of my hobbies is to be a somewhat known stage & exotic dancer / dance instructor in VR using 5-11pt tracking (depending on my mood and production quality). The number of "features" that get in the way, and lack of representation for VR dancers and performers has been frustrating, especially as the world moves more and more towards 3-point robot avatars.

Actually had a chance to provide my perspective to a FAANG Research Group, but I was laid off (and they were disbanded).

Sohcahtoa82• 3 years ago

I can't see VR becoming useful at all for general productivity. AR? Maybe. It'd be cool to have more monitors without taking up space. But even in that use-case, the resolution of current AR/VR tech isn't enough.

Even if you did find a way to make VR work for you at work, current headsets are just too heavy to want to wear them 8 hours/day.

They're still incredible for gaming, though.

burnte• 3 years ago

Replace "VR with "cars", and say all this in 1923. Same thing. There's a lot to be done, few domains of knowledge are completely mined in 30 years.

h2odragon4• 3 years ago

1923 cars had about the same wheelbase, and steering gear variety, and differentials... the basics were all worked out and agreed on.

pavlov3• 3 years ago

Really great article. I’ve never heard before that VR headsets produce collimated-light images, but the explanation makes a ton of sense.

intrasight2• 3 years ago

I found the explanation lacking. It just states that collimation is the case without explaining why. The why is the limited optics of HMDs. In the physical world that those optics are trying to simulate, photons can leave from any point in space at any angle.

lamename• 3 years ago

Is there a graphical explanation of the perceptual phenomenon the author is describing? I think I understand the physical stimulus he describes, based on diagrams in papers, but I havent found anything explaining this part:

So translating the pupil around doesn't change the angle of the light rays, which means the image doesn't move.

twic• 3 years ago

photons can leave from any point in space at any angle

Not really. They can leave from any point on a surface at any angle. Once they're on their way, they don't change direction. If a surface is an infinite distance away, then by the time they reach your face, the photons from that surface are all travelling parallel to each other. That's what collimation is simulating: looking at objects an infinite distance away.

Why? As the article says:

"But try this experiment. If you can find a fixed test pattern, put that on the HMD's screen, then pull the HMD away from your face a little, and move it side to side by an inch or so (don't rotate it, just translate it). The image stays fixed in space! It’s a really strange thing – you’re moving the physical box, but the image inside it doesn’t move. That’s the power of collimated-light images, and it's still very unintuitive to me even though I understand the physics and maths very well."

Just how, when you look at a surface a great distance away, its appearance doesn't change if you move your head a few millimetres from side to side.

"Of course real VR lenses are not perfectly collimated for various reasons. But they're close enough that getting the IAD (optical axis separation) adjustment perfectly right is not as important as it appears at first."

Collimation means that the device is robust to the exact distance between the user's eyes. That is very useful.

bullen2• 3 years ago

When I built my VR MMO in 2014 I experimented with very wide camera separation, it's very particular; makes things look very small and distant but they pop out much more. Like 3D on steroids + tilt shift.

I also experimented with twice the rendering speed to get as recent as possible sensor reading before the Valve/Oculus warping became standard and it was VERY good for nausea. So 120/150 FPS respectively on DK1/2.

I'm looking forward to pancake lenses, if they remove godrays so I can play my game again: http://aeonalpha.com

But I read there are a lot of tradeoffs with them too.

So far DK1/2 lenses with cups that you could change and adapt to your diopter where the best optics VR has had (but I got lucky with IPD and one cup on DK1 being my exact diopter!)

I wish one actor made super tiny glass lenses just so we could see how that would work!?

Here is some more experiments: http://aeonalpha.com/fov.html

nrp2• 3 years ago

We had a somewhat undocumented 120Hz mode on DK2 where both eyes would get the same image (meaning everything was at infinity distance), but I’m not sure if any developers used it.

bullen2• 3 years ago

But then you get no depth perception?!

syrrim• 3 years ago

Could you render very close objects twice, far objects once, then composite the image for each eye?

moron4hire4• 3 years ago

It's funny. In my experience, stereo rendering is not a make-or-break issue in VR. I mean, it depends on the app design, of course, but a lot of apps can get away with monoscopic without most people noticing.

I had a regression in an app I was building where it was rendering in a monoscopic mode and it actually took me a long time to figure out what was "wrong" with the app, but apparently, I was the only one who thought it was wrong. It went for at least a month before I noticed a problem and another month before I eventually had the time to diagnose it. All during that time, nobody made any comment about it, people were enjoying the app.

I eventually figured out that you really only notice stereo rendering in modern VR headsets if you have a lot of near-field objects with which to interact and you interact with them with direct hand gestures (either hand tracking or controller bumping). So, you need stuff to be within a meter of the user for them to really notice "this isn't right".

There is a gap past a meter and before 10 meters where it's hit or miss if anyone is going to notice the lack of stereo rendering. The more the user is moving around, the less likely they are to notice.

And then, anything past 15 meters, I don't think there are any VR headsets on the market (other than maybe the Varjo VR-1, but I've never seen one myself) with fine enough pixel pitch that anybody is going to notice the lack of stereo rendering.

Now, that's just "where does stereo rendering take effect"? Part of what I think makes VR good is the ability to get up close and personal with the environment, to manipulate things in your hands, so typically I think "good" VR apps are going to need stereo rendering, but it was still surprising to see that it's not a universal issue.

two_handfuls3• 3 years ago

I really like the suggestion of looking at the HMD’s faceplate and having it measure your IPD. I think that’s a great idea!

As they say, it’s hard to adjust it optically by moving the lenses because there’s a wide range of values that give a focused image - but the wrong values will result in nausea over time.

javchz• 3 years ago

I recently went from a quest 2 (limited 3 IPD settings) to a pico 4 (that has gradual fine adjustment) and makes a huge difference. Specially in the headache and nausea department.

Even with pancake optic artifacts, just that setting alone makes a huge difference in don't feeling dizzy in rapid movements.

thendrill2• 3 years ago

The varjo xr headsets use the eye trackers to auto adjust the IPD mechanically the moment you put the headset on.

The problem is it takes a second or two and it fails.... Often, so you need to repeat the (take on and off). It makes development and multi person demos into a pain of "put it in and off again it will work"

nomel• 3 years ago

The Quest Pro does the same, with the fit adjustment app.

two_handfuls3• 3 years ago

Yes I agree something like this is probably the future, once it’s more reliable and more affordable.

For now only expensive headsets have the hardware for this. The solution proposed in the article works with the hardware in more affordable headsets which makes it intriguing.

VR optics and why IPD means too many things (2019)

Related Stories