How Magic Leap Is Creating an Alternate Reality

http://gizmodo.com/how-magic-leap-is-secretly-creating-a-new-alternate-rea-1660441103

by ghosh • 12 years ago

230 110 12 years ago

I think Gizmodo is a source HN's algorithms penalizes, which is a shame in this case. It's an interesting article with some good details.

syoc• 12 years ago

This. I refuse to click on gawker links but I would love to read the article. It would be nice with a pastebin / imgur link when people post things from gawker.

tempestn• 12 years ago

That sounds awesome. Also much less tangible than the Rift, obviously. But with that kind of funding, obviously some smart people believe they've got real potential. Colour me excited.

bake4• 12 years ago

This seems like the future for Google Glass -- those articles a few days back about how the program is waning / losing steam seem to overlook Magic Leap, which feels more like Google doubling down.

baldfat2• 12 years ago

I think Google Glass (Aka notifications on your eye glass with a camera and augmented reality are apples and oranges.

For notifications and wearables glasses was an idea that has proven to be a bad choice for social as well as practical reasons.

Augmented reality is well adding to our reality and I believe what people really want and NOT virtual reality. I can image Amazon Echo with a butler image would be a HUGE hit as opposed to the audio only option.

bake4• 12 years ago

If it works, Magic Leap will mark an entirely new era in machine-human interface. It could eliminate the need for screens, keyboards, and even other input devices like light switches and thermostats by turning any surface into a tactile interface. Why actually attach a piece of hardware to the wall when I can just render it there whenever I look in that direction? Why carry along a physical keyboard when I can just 'project' one on any desktop at hand?.

There's also an interesting health & wellness aspect to this -- could this be a way to free people working digitally from sitting at desks in front of screens all day?

This is, in my opinion, the most exciting technology in the popular press right now.

sjtrny• 12 years ago

Why carry a keyboard? So you can touch type. You would have to be looking at the simulated keyboard to use it with this technology.

bake4• 12 years ago

Good point, but some touch typing does happen with the keyboard in the field of vision, just not as the focal point -- when typing on a laptop and looking at the screen, for instance.

wingerlang2• 12 years ago

Rendering a keyboard on a desktop would suck to write on though.

bake4• 12 years ago

Any worse than on a touch screen?

BjoernKW• 12 years ago

There's also an interesting health & wellness aspect to this -- could this be a way to free people working digitally from sitting at desks in front of screens all day?

Probably to the extent technology has freed people from commuting to the office every day. This is more a matter of culture than of technology alone. If we don't make a push to fix broken habits and change detrimental work culture these things will never really change.

7952• 12 years ago

A lot of old technologies tend to hang on within a niche that they are optimal within. We still have radio, cinema, and telephones even when each of those technologies has a better more modern equivalent. I don't doubt that AR will find a niche but I doubt it will be replacing 10$ keyboards or 50c dials on thermostats. It is fantastic that Magic Leap has moved beyond the horribly mundane "helping people find a restraunt in San Fransisco" model that Glass seemed to adopt. Fantasy AR overlays sound ridiculosly addictive as it would allow contsant enveloping escapism.

orbifold• 12 years ago

I'm not sure if Fantasy AR overlays are really all that compelling. Unless they make significant progress in AI as well, most likely those will be entirely predictable and boring after a while. Although who knows maybe there are enough people around who want to have a virtual pet dragon. Most of the real world outside is entirely unsuitable as a canvas for any kind of interesting movie like experience. Traditional movie story telling artfully uses both cuts in time and transitions in space to tell a compelling story, neither of which is readily available.

blaze33• 12 years ago

There's also an interesting health & wellness aspect to this -- could this be a way to free people working digitally from sitting at desks in front of screens all day?

I dreamed of a future where I could have magic leap contact lenses while hiking in the mountains. My brain being quite active when walking or running, that would be an awesome way to do productive and/or creative work.

tegeek2• 12 years ago

This November is pretty heavy for geeks & Singularity lovers (I'm one of them).

First we saw a man-made machine landing/crash-landing 500 millions KM from Earth.

Then HN discussed this [1] which is IMO one of the biggest machine-learning demo I've ever seen (other than Self-driving cars).

And now this futuristic tech which is aiming to eliminate the screens altogether. (As some of its use cases)

If Magic Leap delivers her promise within next 2-3 years, then we are already ahead than Singularity timeline [2].

1. http://cs.stanford.edu/people/karpathy/deepimagesent/ 2. http://en.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzwei...

melling• 12 years ago

How can we be "ahead of schedule" when we still don't have the first couple tasks from a decade ago? Dictation, for example, is pretty good but it's not a solved problem.

* Translating telephones allow people to speak to each other in different languages.

* Machines designed to transcribe speech into computer text allow deaf people to understand spoken words

http://en.m.wikipedia.org/wiki/Predictions_made_by_Ray_Kurzw...

tegeek2• 12 years ago

> Translating telephones allow people to speak to each other in different languages.

The technology is there to build this kind of device, I agree not 100% accurate but still practical enough to be used in real life. http://en.wikipedia.org/wiki/IraqComm

> Machines designed to transcribe speech into computer text allow deaf people to understand spoken words

This too, the technology is there to build a device. I would say we've much more powerful devices in form of smart phones than just doing 1 or 2 things. The talko app on iPhone is actually much more than just speech to text converter. http://www.talko.com/

I would also like to say that Singularity predictions doesn't employ that the technology will be so mainstream that every person will be using that. But even if we can build a technology in some lab, it still becomes an event on Singularity timetable. I've yet to see any off the mark prediction from Singularity timeline.

angersock3• 12 years ago

The funny thing to me is that this is quite possibly already solved, and it's just being fed into a big data center in Utah.

tb10010• 12 years ago

I'm co-founder and research director of a mobile AR software company, Zappar.

I blogged on Magic Leap a few weeks ago, and came to some of the same conclusions as the author of the gizmodo piece - namely that a device that can be like Google Glass and Oculus and everything in between could be a real game changer.

I chose to call the concept "Controllable Reality" (I make no claim that I've invented the concept or the term!): http://www.zappar.com/blog/google-glass-magic-leap-and-the-i...

Edit: I also talk a bit about the requirements for an AR experience, existing approaches to AR HMDs and the big problems to be tackled on the road to consumer adoption. I've got a PhD on the software side of AR but am not an expert on the hardware, but have sufficient knowledge to hopefully offer an interesting perspective.

I'm new to HN too, so let me know if I should be posting this as a separate item or not!

Simon

angersock3• 12 years ago

Erm, what's wrong with the well-understood term "Augmented Reality"?

tb10010• 12 years ago

It's more about capturing the spectrum from AR to VR in a single device. With controllable opacity (which I argue is required for a truly useful AR display) you also have the option to deliver purely VR experiences by blocking out the real world completely.

seren2• 12 years ago

I am a bit puzzled by the names of Austin Grossman, Dave Gibbons and Andy Lanning being cited as employee (whatever that means).

Are they so confident in the technology that they have started to heavily invest in the content? Is this a way to position themselves as an entertainment company rather than a tech company?

Tangokat2• 12 years ago

I agree with this. On top of already doing unbelivable disruptive technology they are also doing several video games, feature films and "being in the same room as your favorite band" content?

Trying to do everything themselves seems overly ambitious. Then again I doubt they get funding like this without something provably awesome and a solid plan to move forward.

rondon2• 12 years ago

Two things.

1. An elephant on a flower isn't going to get someone to pay $500-$1000 for this hardware. You need very high quality content to get lots of consumers on board.

2. Moving from current cinema production to VR production is going to be a huge learning curve for a lot of artists. You need to create best practices and examples of how it is done by artists.

_almosnow• 12 years ago

Those sound like real obstacles. Content is always a big problem, but because of the amount of money that has been poured into this and the people/companies backing it up, I'm pretty sure that they already have that one solved.

dcre• 12 years ago

They have so much money now, they can heavily invest in anything.

arethuza3• 12 years ago

By far the best novel I've seen that looks at a near future (2025) world where most people spend most of their time using augmented reality is Vernor Vinge's Rainbows End:

http://en.wikipedia.org/wiki/Rainbows_End

dmd2• 12 years ago

It's also a DRM-dystopia.

mrfusion• 12 years ago

I don't remember that part. Can you remind me?

scoot• 12 years ago

Interesting! The plot device reminds me of a dystopian short-story, in which the perfectly ordered world the protagonist inhabits ultimately transpires to be retinally projected image seen through government prescribed and mandated glasses that are permanently fixed to all citizens from birth.

Only when he fails to heed warnings to visit his optometrist to have them adjusted does the reality come to light.

kriro• 12 years ago

Whenever I try to get motivated to work on VR, CAVE and similar projects I reread "Ready Player One" and secretly that's the world I'm building towards :)

http://en.wikipedia.org/wiki/Ready_Player_One

dbish2• 12 years ago

Great read. Love the Oasis, although if you want a more in depth read you should check out the Otherland series. Very long, but great representation of what a fully immersive VR world would be like.

freehunter3• 12 years ago

I couldn't get into Snow Crash because it seemed very overdone, seemed forced (the hero/protagonist's name is Hiro Protagonist for crying out loud). Would I be able to get into Ready Player One?

neuromancer2701• 12 years ago

I loved Rainbows End. Recently I read Daemon/FreedomTM, and it paints a very different picture. But this technology could be used to create the Darknet the alternative web/economy in the books.

http://en.wikipedia.org/wiki/Daemon_(book_series)

freehunter3• 12 years ago

Daemon was a game changer for me. Ever since I've read the book (recommended by a colleague), I've dreamed of what could be possible.

That book (along with a small handful of others) made near-future scifi my favorite genre. Any other suggestions?

pavel_lishin• 12 years ago

I think that the HUDs in Daemon/FreedomTM are the least necessary to the possible future Suarez paints. Being able to see virtual stuff is impressive, but having a stable digital currency, an actually democratic political system, the manufacturing technology required to create self-sustaining local polises, etc., etc.

josephpmay2• 12 years ago

Daemon/FreedomTM was what first got me interested in Augmented Reality back when I was in Middle School. They're excellent reads.

tragomaskhalos• 12 years ago

Day N: this tech goes to market.

Day N+1: the first pr0n application using it appears.

hyp03• 12 years ago

It sounds like they've got some trick that gives some experience of real objects - probably far from perfect (hi-latency, lo-res, etc); but that's a good thing for a company. It means they have somewhere to go when competitors catch up.

Maybe opaque and depth of field really fool your eyes?

As a comparison, consider how surprising it is that animation works: line-drawn cartoons look like real objects, animals etc; secondly, that if you just change the images at a mere 24 fps, they come to life (the meaning of animation at the time).

cma• 12 years ago

One problem with low framerate in AR is things come "unglued" when you move your head.

(There are some workarounds: low-persistence, asynchronous timewarp)

egypturnash• 12 years ago

The threshold is lower - a hell of a lot of hand-drawn animation is done 'on twos', with each frame shot twice, for an effective framerate of 12fps. 'Going on ones' is mostly reserved for motion that has to match a panning background (panning the whole screen at 12fps looks terrible, and so does someone whose feet are siding back and forth with respect to the ground), or for really fast motion. Or occasionally for really smooth, subtle motions.

In my experience 10fps is when cartoon images start to break down into "a sequence of drawings" rather than "moving images". This is right on the line for me; I'm not sure how common that is for people who haven't trained their eyes to the point where they can pick up on a single frame being shot 3x instead of 2x.

I mean, yeah, 24fps looks better, 30 or 60 fps is even better, and the maximum frame rate of the human eye seems to be anywhere between 255fps to 1kfps (I'm getting a lot of conflicting info from a quick google for that), but 12fps is pretty solidly acceptable for moving drawings.

(I'm a former animator.)

hyp03• 12 years ago

Maybe very low-quality AR can be convincing, provided you're stationary (i.e. no panning-like effect)?

Relatedly, Carmack found some serious latency problems with Oculus Rift, but only with some particular combinations of object movement and changing viewing direction. He's thinking of fast video games with lots of movement. Just having a dragon floating in front of you is quite a lot more constrained.

Though I meant it as a general observation, i.e. animation is surprisingly effective, if you do it the right way; perhaps AR will be too, if it's done in (so

hyp03• 12 years ago

\tangent I've always loved the idea of eye-tracking. It helps the whole graphics stack, because don't need to render the whole "screen", just the exact part you're looking at. And you only need render the center of it in sharp detail (what impinges on your hi-res fovea); the surroundings can be blurry and even monochrome (no cones outside the fovea, only rods!).

Unfortunately, because eyes saccade (twitch) extremely fast, latency must be ridiculously low or it would send you insane (figuratively).

xorcist• 12 years ago

It would also be impossible to use for more than one person at a time.

aaron695• 12 years ago

One assumes the big players are seeing through the gimmicky whales and seahorses and can see a wider more functional future.

Pretty things cost huge amounts of money to create. Making them 3d and interact with your environment just adds to the cost.

jcfrei• 12 years ago

I wonder how such a "display" of the Magic Leap is controlled. Could it be that it's no longer a rasterized display? Maybe rather than coloring pixels on a plane, the developer is required to send display information of a volumetric object like in a volumetric display: http://en.wikipedia.org/wiki/Volumetric_display

tb10010• 12 years ago

In general you just to provide the depth map too. You're still only rendering for one viewpoint (well OK, one for each eye) so the display just needs to simulate the depth and colour of the point nearest to you along any ray.

tb10010• 12 years ago

...distinct from volumetric displays which support multiple simultaneous viewers so need to actually render in terms of voxels.

Combining that with tracking depth cameras and sharing the map of the environment over the network would let multiple co-located users see and interact with the same 3D object (much like with a volumetric display) but the actual rendering of each user's view is decoupled.

Combining this with eye-tracking, as \hyp0 mentions, takes this to the limit - you don't need to construct a "holographic display" that renders super high resolution light fields of rays in controllable directions to all the infinite possible viewpoints around the display. Instead just stick it on your head, locate the users in a shared depth map, and just render the slice of that their eye needs to see right now. In terms of CPU/GPU/bandwidth issues it's much more plausible with today's technologies. The display side is still challenging!

coldcode2• 12 years ago

People often talked about Steve Job's reality distortion field but Apple shipped real products that did change the world. I will believe in a magic leap when I see it appear, not when I hear all the hype machine output.

remon• 12 years ago

I agree but the level of funding and big names involved do seem to indicate there is some substance there as well. I'm fascinated.

baldfat2• 12 years ago

We are talking about something new in the works. Just like everyone else besides Apple they need to get the word out BEFORE the product to get funding etc.. I don't see how Magic Leap deserves a remark alluding to vaporware???

Tloewald3• 12 years ago

I think it's odd that they're doing this kind of PR push when their product is so tenuous. If you look at Carmack's keynote and the low level detail of how Occulus is struggling with latency and precision in head tracking, the idea that these guys will tackle a whole swag bag of deep problems in some kind of predictable time frame is pretty ridiculous. My guess is they've made promising inroads in the core display technology (lcd occlusion plus fiberoptic scan projection) that is very impressive on its own, but everything else they've got is merely state of the art.

coldcode2• 12 years ago

Without PR push you don't get the big bucks. Apple for sure today doesn't need that so they can afford to stay quiet.

tfinniga2• 12 years ago

PR push? This article doesn't seem like a PR push at all. It seems like someone scraping as much information out of as many sources as possible.

If it was a PR piece, they'd have animations and pretty renders about how all the things work and where they're going. Instead there are patent diagrams and quotes from job listings.

This seems more like journalism.

dannyr• 12 years ago

PR to me means the company itself is involved.

Magic Leap seems to be keeping its silence.

What they are doing is the opposite of a PR push.

dmritard96• 12 years ago

Agreed on some level. It leaves me with, well either they have some awesome breakthroughs at each point along the critical path or that they are so close that its possible that they overcome whatever is left.

acqq• 12 years ago

I also started to read the article and one every third sentence thought "if it were true, Carmack would already work on that and tell us all."

It does sound suspicious to assume that they magically solved these hard problems.

aetherson• 12 years ago

Yes! People's take on Magic Leap is so credulous. It's like they believe that this company is going to produce something with all the virtues of the Oculus Rift + Google Glass and none of the problems with either on a fraction of the budget and in a fraction of the time.

It just obviously won't happen.

oh_sigh4• 12 years ago

Yeah, I got a really weird feeling in the video of Graeme Devine from the bottom of the article. He introduced himself as being from magic leap, and kind of paused, as if waiting for people to break out into applause for the company that no one really knows anything about.

Sophistifunk• 12 years ago

cough Segway cough

thewarrior• 12 years ago

Imagine going to the Eiffel Tower in Paris.

On top of all the real people walking there , you'll have virtual people as well who are all in their living rooms yet still interacting and conversing with everyone as if they were really there.

Everyones Magic Leaps would be synced to show the same composite reality.

You could have virtual assistants to show you around the city , explaining the various attractions or just to give you some company.

Ofcourse you'd need a gigantic network of overlapping 3D cameras to support such a feat but we won't let concerns like feasibility get in our way. I dreamt of this once and now I'm really excited that I might get to see this within my lifetime.

mgirdley• 12 years ago

The Seqway had similar amounts of mystery and hype years ago.

ohsnap• 12 years ago

Sounds awesome, but if it is as gizmodo describes it feels like one of Google's moon shots. We are just barely starting to see good VR... combining a real and virtual world together is another order of magnitude of difficulty.

I'd love to be proven wrong but don't see this for many years.

sixQuarks• 12 years ago

Magic Leap has Brian Cox and the team behind the movie Gravity working with them as well. This is impressive if they can pull it off: http://www.bbc.com/news/entertainment-arts-30056232

Tangokat2• 12 years ago

Pretty interesting quotes from a guy like Brian Cox:

"It's the premiere of a technology that allows you to put digital images into your field of vision directly," he said. "I saw the prototype in Miami a few months ago and it's stunning."

"It is going to be transformative technology, there's no doubt about that."

The experience will "disturb" audiences and put them "off balance", he predicted. "That's what it did when I saw it demonstrated."

The article also says that a demonstration will be made as a part of the Manchester International Festival next July.

peterwwillis• 12 years ago

Maybe I should have stayed at MAKO a little longer for that payout... derp.

hemancuso3• 12 years ago

By all accounts, they are working on some sort of device that projects a light field onto the retina. But that is strictly additive. You can't darken anything. This image you see of the elephant in your hand - no way to do that. At best it'd be rather transparent.

Unless there is some aspect of vision that somebody here would like to chime in on [locality on the retinal sensor allowing to perceive darkness by lightening up other areas?], color me deeply skeptical. It seems like "fullscreen" Google Glass, at best.

mach5• 12 years ago

the article claims they are working on/have an LCD 'masking' layer can black out the area behind the projected light field

tb10010• 12 years ago

Yeh, the controllable opacity is really important to the utility of the device IMHO.

I've yet to study the linked patent so can't say if that solves it but it's not as simple as just a layer of LCD pixels, as they would not be in focus.

It's sort of the same as the light-field generation problem - to generate images at arbitrary depths you need to control both the colour and direction of light rays. To generate sharp occlusions at arbitrary depths using a single 2D "masking" layer will require the layer selectively passing or blocking rays depending on their direction.

The patent mentions variable diffraction gratings, which I suspect might be part of the solution to controllable opacity based on ray directions, but I don't know enough about optics really to say for sure.

bri3d• 12 years ago

https://www.google.com/patents/US20130128230?dq=inassignee:%...

This patent describes the "occlusion mask device" which is supposed to do exactly this - darken light coming from certain angles to grant the perception of blackness.

hemancuso3• 12 years ago

Seems like a screen in front of your eye that blocks out light.

tb10010• 12 years ago

A big unanswered question I have is what the level of transparency is going to be when the display is inactive. For me it needs to be basically 100% to enable an "always worn" piece of hardware. The patents around the lightfield waveguides suggest polarization is used to allow independent optical paths for rays from the real world vs rays coming from the virtual image. Again, I don't know much about optics so wonder if there's an electronic way to have that polarisation turned off for areas without any virtual content, or if there's always going to be a darkening effect on the real world.

seren2• 12 years ago

Given they have hired creative writers, filmmakers, etc, I don't think they want to create an "always on" device but rather an entertainment device you use at home. See also the article about Brian Cox describing something which sounds similar to a Planetarium like experience on BBC.

DontBeADick• 12 years ago

They addressed that in the article that you're commenting on but didn't read.

hemancuso3• 12 years ago

Well that's not fair. The "occlusion mask" is referenced in a patent application, but basically is just a LCD that blocks out light that sits in front of your eye. re: fullscreen google glass, at best.

angersock3• 12 years ago

Meanwhile, in the land of not vaporware:

https://www.spaceglasses.com/

tb10010• 12 years ago

In the land of "not really good enough". Additively mixed at a single focus depth plane, not dissimilar to the hacks people have been using in research for AR for a long time.

It does however really annoy me the amount of press Magic Leap have managed to get without saying anything public at all. For me it's an opportunity to think and talk about the properties of an ideal AR HMD and why it might be an interesting device. I have no idea how close Magic Leap are going to get to that ideal.

cardigan• 12 years ago

"projects light onto your eyes"

Pretty sure my laptop does this :P

kordless• 12 years ago

It's the inverse of the camera demo on the page though, so that means they have figured out how to project light with the lasers down an optical fiber. Stick that nearly transparent fiber pointing straight at your eye, in your eyeglasses, say, and you have a way to paint on your visual input areas.

josefresco• 12 years ago

$542 million! I'm used to big numbers from following the tech startup world but this seems huge. How many other half-a-billion dollar investments in tech are there?

Doing some Google research, I found some lists of top VC deals:

http://www.entrepreneur.com/article/233632 http://venturebeat.com/2012/12/28/top-venture-capital-deals-...

Looking quickly at the numbers, it does seem that this deal was relatively large.

trishume• 12 years ago

If this technology is as powerful as it claims to be, I could easily see Magic Leap eclipsing Apple as the most valuable company in the world as long as they hold those patents.

I suspect their strategy for finding investors is like this: 1. Potential investor is incredulous about the company's value 2. Investor gets a demo of their technology, and it's mind-blowingly amazing. 3. Investor asks them to name any price and gets out their checkbook, knowing that this has the potential to be the biggest invention of the century.

oh_sigh4• 12 years ago

Considering FB bought oculus rift for $2B, $542M seems about the right ballpark for this line of technology.

androidb• 12 years ago

Can't believe anyone here didn't mention this but I'll be the immature one. Imagine porn on this device. I mean just, you know, all the Scarletts and Jessicas.

fidotron• 12 years ago

Or "X-ray" type applications. Given advances in computer vision and stuff like Kinect that really doesn't seem far off.

anigbrowl• 12 years ago

It's not immature to make the observation. Porn has been a driver of new media technology throughout history - Magic Leap reminds me of nothing so much as a portable kinema viewer.

thinkstorm• 12 years ago

If a "lightfield" is projected into your eye, how do you cope with (additive) light input from the real world?

if it is real, will there be a law to tag 'virtual' things so i don't accidentally kill a real person when I thought that was part of my game? You know how congress or the senate would design that one ;)

How Magic Leap Is Creating an Alternate Reality

Related Stories