Inside Look at Valve's New Lighthouse Tracking System

http://www.hizook.com/blog/2015/05/17/valves-lighthouse-tracking-system-may-be-big-news-robotics?ref=HN

by beambot • 11 years ago

143 35 11 years ago

MrRadar• 11 years ago

There's some discussion of this article on r/oculus: https://www.reddit.com/r/oculus/comments/36cli8/valves_light...

Nexxxeh2• 11 years ago

I wonder how it'll compare to the system Technical Illusions uses for castAR's tracking. Especially given the divergence of the two teams.

(Technical Illusions being the company of Jeri and Rick, who left Valve taking the Augmented Reality tech with them.

Jeri Ellsworth basically built the Valve hardware division. Valve worked on AR and VR simultaneously and then canned the AR stuff. Gabe let Jeri take the AR stuff when that happened.

An interesting story that I've oversimplified. I think it was on an AmpHour podcast but don't quote me.)

theschwa• 11 years ago

My understanding is that technical Illusions tracking system is tied to the retro-reflective mat. The glasses see some LEDs on the mat and use that to determine their position and orientation. So the tracking is limited to the glasses being in view of the mat. If you turn around, you're no longer being tracked.

The Lighthouse system, on the other hand, is a full volume tracking system that can be expanded to cover any volume of space. So I could walk around a room and no matter where I am or where I'm looking, I can determine my location and orientation.

JoshTriplett• 11 years ago

That's not a limitation of the hardware; castAR has cameras and IMUs, and it could orient itself via any identifiable reference point, which need not be the mat. So this is more a problem of software than of hardware.

Nexxxeh2• 11 years ago

My understanding is that technical Illusions tracking system is tied to the retro-reflective mat. The glasses see some LEDs on the mat and use that to determine their position and orientation. So the tracking is limited to the glasses being in view of the mat. If you turn around, you're no longer being tracked.

Yeah, although to clarify, it's not the mat per se, it's the tracking marker. (Seen here, bottom picture, as it's the same one for the wand: https://www.kickstarter.com/projects/technicalillusions/cast...

There's a wand PCB and a wand on there too, they aren't part of the marker.)

The tracking marker is also covered in retro-reflective material, but it's not part of the mat itself. Relevant because there are clip-on adapters to turn the castAR goggles from AR to VR, and you don't need the mat for VR.

It may not be the final version, as the tracking system has gone through a few upgrades and tweaks already according to the updates.

rasz_pl3• 11 years ago

ah, Jeri mentioned standalone tracker diodes available, you could put them all around your house and use Castar just for pose/location estimation.

IshKebab• 11 years ago

That's actually brilliant. I assume you then need two lighthouses so you can do triangulation and get the position of your sensor rather just the angle to the lighthouse.

gallamine3• 11 years ago

You would be able to get some distance information via the modulation on the signal. If your photodiode response was able to pick up phase changes, you can estimate distance. The light arriving at closer diodes will have a different phase than the further diodes. The article mentiones "MHz" intensity modulation on the does which gives you several meter wavelength.

beambot5• 11 years ago

I think your conceptualization is a bit off.

Time of flight as you describe could work at the transmitter, but could not work at the Lighthouse receiving photodiode. To measure time of flight, you need to know (very precisely) when the light was initially transmitted. The synchronization flash is insufficient for this, as an error of just 1ns results in a 1ft error -- modulation or not. Even with modulation, the receiving photodiode doesn't have another signal with which to compare for ToF measurements.

Normal laser rangefinders are transmitting the light (modulated), and looking for the reflected return (modulated) at the same location as the transmitter. They use a PLL to determine the phase difference between the TX and RX (modulated) signals -- since the transmitter has both versions readily at hand. The phase difference corresponds to a time difference => distance. Note that the "reflector" (where the photodiode sits in Lighthouse) is not part of the equation.

In other words, laser rangefinder ToF measurements require coherent demodulation at the location of transmission.

titanomachy• 11 years ago

MHz intensity modulation gives you several meter wavelength

What do you mean by this? The light itself is not at MHz frequency, just the modulation of its intensity. It is infrared, which is ~700nm => 100s of THz.

beambot5• 11 years ago

Nope, a single Lighthouse transmitter can give full position -- at some accuracy. If you use multiple photodiodes (receivers) with a known rigid-body transformation between them, you can infer the receiver unit's full 6-DoF pose.

For example, you could (naively) estimate distance using two photodiodes if you knew that they were separated by 5cm. If the difference in measured angles is small, they're far away. If it's large, then they're near. (The full 6-DoF situation is a little more complex, but the same idea.)

leoc2• 11 years ago

According to Lighthouse's creator:

http://www.reddit.com/r/oculus/comments/35kymh/valves_alan_y...

It can actually (poorly) track a moving object with just one sensor and one base station... We didn't expect this at all, it is not immediately intuitive, but even 2D simulations display this behavior. If you can tell me why, really why, you should apply for a job at Valve. :)

(Apparently 5 sensor hits are required for a proper fix.)

CodexArcanum• 11 years ago

The physics and engineering of light are not something I'm very knowledgeable about, so forgive me if this is naive. Couldn't you reduce the latency further by running the x and y scans in parallel, and using something (e.g. different modulations) to tell them apart? It seems kind of a waste to run the x and then the y in sequence.

hansihe• 11 years ago

If I'm not mistaken, most standard IR sensors have built-in demodulators. They most likely want to keep the parts needed for a receiver as off the shelf as possible.

twic• 11 years ago

This is not entirely removed from how light guns work:

http://en.wikipedia.org/wiki/Light_gun#Cathode_ray_timing

The basic mechanism in both cases is measuring the time taken for a patch of light that is steadily moving from a start point to become visible to a detector, and then using knowledge of the scan rate and the geometry of the detector to calculate orientation.

Valve's system is considerably more advanced than Nintendo's, of course. But then they have had twenty years to work on it (i think that's about two and a half years in Valve Time).

rsp1984• 11 years ago

This sounds very cool but from what I understand from the article I guess it would require direct emitter-receiver visibility, i.e. turning around and facing away from the emitter wouldn't work.

Similarly, if you used several emitters to circumvent above problem you'd run into aliasing issues and you'd have to engineer the system such that different emitters can be clearly distinguished from each other.

The low latency is a big deal though. See-through AR is a particularly hard nut in terms of latency because just the slightest bit of it can completely destroy the experience (overlay images lagging behind real world), unlike VR where all the photons come from one source and some latency is tolerable.

Guvante• 11 years ago

In theory if you could place a second bank of receivers behind your head and had an accurate distance between the two you could simply the problem to just requiring line of sight.

It also sounds like they have a multiple device fix available, the only question being can they devise a way to synchronize two of the devices without requiring the user to do any measurements.

swalsh• 11 years ago

In the article it actually mentions that they already built the system to support it (they are modulating the pulse)

ZoFreX• 11 years ago

you'd have to engineer the system such that different emitters can be clearly distinguished from each other

It mentions in the article that the light is modulated to permit multiple transmitters, so this seems pretty robust.

nemof• 11 years ago

you can stack as many lighthouse units as you want in an arbitrary configuration (which will kill for having huge spaces spanning multiple rooms). i believe the vive headset demo had 2 or 3 units covering 15x15 feet area.

they also have sub millimeter accuracy, to the point that people can pick up physical objects mapped in 3d space, which was how valve demoed picking up the vive controllers whilst you had the helmet on, so you saw the controllers in the 3d space, and they were exactly where they were in meatspace.

nickysielicki• 11 years ago

couldn't you just set them all at different frequencies?

if not, why not?

cma2• 11 years ago

The lighthouses seem to have a 90° sweep. The sensors on the Vive headset (but for some reason not the valve controllers) seem to be in 90° occlusion pits. This means that if the two lighthouses are at opposite corners of a (square) room, for the most part no single sensor will ever pick up both lighthouses at once, no matter where the headset is in the room or how it is oriented.

I think that was the initial strategy, but now they are going with a timing based or modulation based solution, and that's why the controllers, which were developed later, don't have occlusion pits. In this interview with Alan Yates of Valve, he mentions timing, modulation, the role of the LED array, and getting rid of the wire between lighthouses: https://soundcloud.com/hackertrips/alan-yates-of-valve-talks...

cma2• 11 years ago

(actually from a talk today, it's a 120 degree sweep)

Inside Look at Valve's New Lighthouse Tracking System

Related Stories