Lytro Unlock – Making a bad camera slightly better

by Nrbelex • 3 years ago

363 73 3 years ago

joshu4• 3 years ago

I love this. I collect interesting obsolete hardware; I bought a Light L16 recently for cheap but it can no longer update itself, unfortunately. Lytro is on my list as well.

The lytro itself is a lightfield / plenoptic camera. It captures the angle of light coming into the camera as well, which lets you calculate focus AFTER the photo has been taken. Focus is, of course, itself just another computation. http://graphics.stanford.edu/papers/lfcamera/

sacnoradhq2• 3 years ago

-- Media --

https://en.wikipedia.org/wiki/Smell-O-Vision

https://en.wikipedia.org/wiki/MiniDisc

Toshiba Twin S-VHS changer https://youtu.be/aEEYuJw7vFk

https://en.wikipedia.org/wiki/HitClips

https://en.wikipedia.org/wiki/Pocket_Rockers

0.85" / 22 mm HDD used in MP3 players

https://web.archive.org/web/20120718025547/http://www.toshib...

-- Computing peripherals --

https://en.wikipedia.org/wiki/CueCat

https://www.computinghistory.org.uk/det/12416/Logitech-Cyber...

-- Alternative computers --

https://en.wikipedia.org/wiki/Amiga_500

https://en.wikipedia.org/wiki/IBM_PCjr

-- Electronics --

https://en.wikipedia.org/wiki/Capsela

https://en.wikipedia.org/wiki/Heathkit

Grandparents had a Heathkit TV like this with an ultrasonic remote that could also be controlled by jingling keys:

https://www.nostalgickitscentral.com/heath/products/tv/gr250...

== Of course ==

Grandpa's Polaroid 350

https://en.wikipedia.org/wiki/Polaroid_Corporation#/media/Fi...

jacquesm• 3 years ago

Grandparents had a Heathkit TV like this with an ultrasonic remote that could also be controlled by jingling keys

I wished my Sony A7 had such a nice and elegant shortcut for remote video recording. Instead it has a crappy little remote control that eats batteries like candy and that seems to un-pair spontaneously in mid recording session and other such niceties.

joshu4• 3 years ago

I have a cuecat and a pc jr keyboard in my collection, as it focuses on tech innovation failure. A bunch of other items on this list are good additions.

I should probably make a list of my collection. A working juicero. A rolleron. An itanium cpu. A working General Magic device. The iBook I wrote del.icio.us on…

I had capsela as a kid. Hadn’t thought about it in decades. Thank you.

steanne• 3 years ago

librarything still sells and works with cuecats

slopperyslip• 3 years ago

Was curious about a Light L19 seeing how cheap it is now and did some digging around for how to update. This Github project from an XDA thread may do the trick https://github.com/helloavo/Light-L16-Archive

joshu4• 3 years ago

Thank you for this.

icelancer• 3 years ago

Oh wow, this camera! I saw it years ago and never thought about it again. It's such an interesting idea for my field of work that wants variable focus throughout a video shot with high speed photography, but can't get it with the equipment that exists on the market.

fire• 3 years ago

thanks for this, I've been needing to factory reset my L16 but I couldn't locate any of the firmware

d1sxeyes• 3 years ago

That’s a really interesting way of phrasing it I’d never fully considered: of course you are right, a physical lens is just performing predictable mathematical transformations on the light passing through it.

Thank you, that was something of a revelation in understanding how these cameras actually work.

joshu4• 3 years ago

More importantly, focusing a lens is one of many possible transformations. Look up coded apertures. so much is possible.

weinzierl• 3 years ago

"just performing predictable mathematical transformations"

Just linear algebra, to be precise. A lens is just a matrix and rays are simply vectors. Combining lenses is merely matrix multiplication and complex optical systems can be represented by the result, which is a plain simple matrix again. The Eigenvalues tell you interesting properties of your optical system.

I've always enjoyed this part of physics because it is so simple and elegant, yet so powerful.

If you look closely linear algebra pops up in many places.

dreamcompiler• 3 years ago

Lenses compute 2D Fourier transforms. And they do it really, really fast.

https://en.wikipedia.org/wiki/Fourier_optics#Fourier_transfo...

sacnoradhq2• 3 years ago

Oh sweet. I have an original Lytro. Battery is dead so it would need a repair.

TaylorAlexander2• 3 years ago

Ooooh I have an original Lytro and of course it just sits there. I am going to have to try this!

By the way don't miss this video about the failed 755 megapixel 300fps Lytro Cinema camera, a contraption the size of a car with off the charts data storage requirements.

https://www.youtube.com/watch?v=pfjYecJHMRU

obamallama• 3 years ago

If the human retinal density was equal to the foveal density across the whole eye, our optic nerves would be half a foot across.

And if my grandmother had wheels, she'd be a bicycle.

In short: pay attention or get left behind.

ideashower• 3 years ago

The whole segment never fails to crack me up on a bad day: https://www.youtube.com/watch?v=OplyHCIBmfE

carlosdp• 3 years ago

What's wild is that it turns out we were only a few years away from technology that could completely replace the purpose of the cinema camera, with off the shelf GPUs on a normal desktop (NeRFs)

oofbey• 3 years ago

NeRFs are cool and someday will do what the Lytro cinema camera claimed. But today it’s nowhere close. See how Lytro took 10 years to go from lab science to a cinema quality camera. NeRFs have a similar journey ahead of them. Today they kinda mostly sorta work in the lab.

Firerouge• 3 years ago

Shame lytro never made a security video camera variant. Seems like the ideal application, ensuring that anything from vehicle plates in the background to faces in the foreground can be brought into focus.

What sort of fps is the API capable of producing jpegs? It sounds like the larger illum can manage 3 fps natively, but couldn't find specs on the original lytro.

skhr0680• 3 years ago

Ideally a surveillance camera will already be focused to have the entire scene sharp. The Lytro camera had postage stamp resolution as far as I remember so it wouldn’t solve the problem that you are thinking of.

folkrav• 3 years ago

Ideally a surveillance camera will already be focused to have the entire scene sharp

In practice a lot of surveillance cameras seem to be focused to have the entire scene a blurred mess

WithinReason2• 3 years ago

The images produced by Lytro are 1.2 megapixels, so not enough to resolve small detail.

atdrummond• 3 years ago

They had a “DSLR” which was about 4x higher resolution than the original device.

MrGilbert• 3 years ago

I must admit, I have a really soft spot for well-done reverse engineering write-ups like this. Show me logic board pictures, console commands, code and who-knows-what, and I'm happy as a squirrel.

mjburgess• 3 years ago

I like the simplicity and elegance of embedded programming, but I never have much motivation to do any. I've never really been a hardware person (out of choice).

worldmerge• 3 years ago

Very cool! Would it be possible to use this for NeRFs? I’m very new to them but it seems like they use light data heavily, at least it seems Luma AI does.

Zetobal• 3 years ago

It wouldn't make sense to use these for nerfs because of the variable focus... and the low resolution. The depth prediction used by NERF is also miles ahead of the depth maps produced by lytro cams.

ladberg• 3 years ago

I'm not sure if you're already aware of the connection, but the founder of Lytro (Ren Ng) was actually the advisor on the original NeRF paper! I took a graphics class from him in college and it was one of my favorite classes. In addition to the course material it was great to hear about what his lab was working on and future moonshot projects.

olliej• 3 years ago

awww, I finally threw mine out during the house cleaning maybe two months ago :(

UmYeahNo• 3 years ago

Weird. Came here to say the same thing. I let mine charge for a couple days and it wouldn't come back to life, so off to the recyclers it went. Such a shame.

seszett• 3 years ago

I've been intrigued by this camera ever since it came out but I had not checked eBay for a long time so I was surprised to see "$20 to $40" mentioned there.

Well, the cheapest one is actually $200 as far as I can see, showing results worldwide.

ljf• 3 years ago

I saw one here in the UK at a carboot sale last year for £5 and I picked it up and chatted to the owner about it, but knew I had no need for it. On the way home I passed him again and he offered it to me for free, but I politely declined as I already horde enough stuff - so they are out there for a bargain, just need to get lucky I guess.

birdman3131• 3 years ago

Look in completeds. You can tell when this software became known. https://www.ebay.com/itm/144836103271 was one for cheap.

gommm• 3 years ago

Yup, same here :(

WithinReason2• 3 years ago

Could you construct a NeRF from a single Lytro image?

jasonwatkinspdx• 3 years ago

I believe so. The underlying data is essentially 4D.

dTal2• 3 years ago

In all likelihood yes, but it wouldn't do what you likely want. NeRF is better for interpolation than extrapolation, because it can't confabulate things that were occluded in all of the input data. The main lens of the Lytro is small, so it's basically like having all your input images confined to a circle a few inches big. As soon as you tried to reconstruct anything outside that, you'd see gaps.

It might be useful for boosting its mediocre resolution, though.

jonah-archive• 3 years ago

!! I need to check this out. A friend gave me an old Lytro Illum with a bad test firmware awhile back that I never got around to doing anything with, maybe there's some possibility here.

rixrax• 3 years ago

Nice to see old Lytro hardware brought back to life.

When they were launching the company, I was so excited about the possibilities. Maybe naively, I was really rooting for being able to take a photo, and then in their software (and later in Lightroom) be able to slide/brush-in/out points of focus. Alas, all we got were these gimmicky square pictures that you could click to bring various parts to focus.

nathancahill• 3 years ago

Yeah, I think the concept is cool from a physics standpoint and the software didn't do it justice. If it had launched after iPhone shipped Portrait mode I think it might have angled more in that direction. My sense is that it came to market too early.

AstixAndBelix• 3 years ago

As always, a company makes a very interesting piece of hardware available but locks it down with proprietary software so when it dies almost nobody can keep using it effectively.

At least these project of reverse engineering the file format or even the firmware can bring new life to these quirky cameras

donatj• 3 years ago

I was just tweeting the other day about how my Lytro was really fun for a couple days and jogging through focus was neat but after the initial glow it was just an overpriced camera that gave me underwhelming 4 megapixel photos as the final product.

I had always hoped for better software to come along. I’ll be curious to see if it can find a second life.

daniel_reetz2• 3 years ago

Author, if you're reading, a really cool thing would be to create a small array of these cameras (4 of them) side by side.

The very wide baseline should allow you to enhance/deepen the refocusing effect and create a wide format image. I've always wanted to do this.

Frost1x• 3 years ago

You can also extract depth information from each cameras lens array. In theory, if you combine that capability on a per camera basis with photogrametric and/or stereo imaging techniques, you should be able to get a fairly refined depth map of a scene passively if the scene is static or you have well synchronized shots. Something I also wanted to explore but never had time to do.

At the time, I always considered passive ranging to be a potentially more valuable feature of lightfield cameras more so than refocusing images.

PaulHoule• 3 years ago

There is software to make point clouds and such from the Lytro

http://www.plenoptic.info/pages/software.html

I take stereograms with a Qoocam Ego, I have thought about getting a medium size Lytro, (the one that looks like a DSLR.)

daniel_reetz2• 3 years ago

My point was that the depth information increases with greater stereo baseline (distance between lytros in this case).

Depth maps from light fields were a topic Disney Research was publishing on while I was there. Great stuff, highly recommend checking out their siggraph paper.

matsemann2• 3 years ago

Killer feature of this new technology was the ability to refocus the image after it was taken! (...), the camera was trying to solve a problem that didn’t exist

A related camera technology, but which I've found great to use, is 360 cameras. The ability to re-frame (not focus) the image as I want in post is so great. No longer do I have to spend many attempts positioning a camera perfect, just shoot and later "point" the camera correctly. For instance when filming ski videos, it could take multiple attempts getting the angle correct to get exactly the person+jump in view. Now it just works.

lostforwords• 3 years ago

I worked in scientific imaging and when Lytro came on the scene it showed a lot of promise. It brought the promise that we would be able to clearly capture and measure across multiple focal planes in a single image. Lytro focused on the consumer audience and never produced a product suitable to scientific imaging. You'll see Raytrix is still around in this space as they did embrace the scientific market.

eyesee2• 3 years ago

I worked on 360 cameras / software for many years. Looking forward to the day when an article like this comes out for one of the products I worked on.

When you’re in the heart of it it’s so easy to take pride in the technical challenges you’ve overcome, but completely miss the realities of the marketplace.

matsemann2• 3 years ago

I really love my 360 camera. Use it everyday. I fill my 128 gb memory card multiple times a week. I use it on my helmet when biking for work. On a selfie stick when skiing so that it looks like I have a personal camera man following me (example https://www.instagram.com/reel/CnFtVsMJd39/?igshid=YmMyMTA2M... )

But yeah, they haven't taken that much off compared to traditional action cams. The frame rate and resolution after reframing is the blocker for many. And hard to do that with the current available sensors it seems like, and no one right now wants to gamble on spending much on R&D for launching the next gen.

jbandela12• 3 years ago

On a side note, do you think think type of light field photography could be packaged in a phone camera? Right now we have multiple lenses sticking out the back of the phone. Could we replace all that with just a single large light sensor?

GeompMankle• 3 years ago

If I correctly interpreted your question as ending in "a single large light[field] sensor], no. Light field cameras take a single field of view and overall entrance pupil and then, given that field of view, they trade resolution for being able to specifically note which part of the entrance pupil saw the object in that low resolution way. As a person who specializes in physical optics rather than computational optics, my extremely biased standpoint is that light field sensors can take one really great camera and make it into a bunch of fairly poor cameras collecting 3D depth information. If you value 3D depth information and the ability to add realistic depth of field blur to a photo more than the 2D resolution and sensitivity/noise properties of the photo, lightfield is for you.

The reason we have multiple lenses sticking out of a phone is to achieve similar high image pixel counts at high sensitivity/low noise across wildly different fields of view (say 108 deg, 69 deg, 28 deg) and wildly different per-pixel angular fields of view in a cost effective manner. A single imaging block capable of seeing a whole 108 deg field of view wide camera image with the per-pixel angular field of view of the 28 deg telephoto imager is generally infeasible in a cost effective way unless you are IMAX.

jbandela12• 3 years ago

Thank you for your detailed answer.

highenergystar• 3 years ago

I wonder if a lytro could be attached to a full frame DSLR and take the 'same' picture as the DSLR, and augment the DSLRs picture with the depth field info captured by the lytro ...

sedivy94• 3 years ago

I was just thinking about Lytro the other day... I was quite young but very excited by the prospect of that technology. I know computational photography and AI have done more for cameras in the past decade, but I'm still hopeful there's a paradigm-shattering advancement in sensor tech.

stuntkite• 3 years ago

I always wanted to get one and use it for depth mapping. I'm so glad someone else had the same idea. Kind of a bummer that the company tanked, but I get it.

gumby2• 3 years ago

I'm really surprised that LF cameras haven't been added to any phones. I'd think it would be valuable for "fixing" photos and perhaps allow computation to replace the LIDAR.

Though it feels like the cameras are in a "pro" or at least "prosumer" race. Every apple camera announcement seems to tout "advances" that are mostly irrelevant to me (who, like I assume most people, pretty much only uses the camera to take pictures of people and pets).

kridsdale1• 3 years ago

The latest iPhones fake this in Cinematic filming mode. They use depth info and a long depth of field, then apply a distance based Gaussian blur to fake a shallow DOF in post. But you can go back and edit footage, selecting other subjects than what was “focused on” during filming.

AustinDev• 3 years ago

Dang, I literally just sent my Lytro to the recycler during my spring cleaning.

Lytro Unlock – Making a bad camera slightly better

Related Stories