Light Field Photography

http://news.kynosarges.org/2014/05/11/light-field-photography/

by cnahr • 12 years ago

58 30 12 years ago

news.kynosarges.org

acous4• 12 years ago

"Holographic Recording ... remains irrelevant to consumers until we have convenient holographic displays to match."

I'm not sure why Lytro hasn't touted this side of the technology... the first consumer holographic display (oculus rift) is due relatively soon.

phreeza• 12 years ago

The rift is not really holographic, it's 'just' stereoscopic. Nvidia is working on light field glasses though, and I believe this might be the future, because it allows for much sleeker eyepieces, at a higher computational cost. https://research.nvidia.com/publication/near-eye-light-field...

acous4• 12 years ago

I believe[1] that the head tracking qualifies it as holographic. That light field stuff looks super interesting...

1. http://doc-ok.org/?p=337

bane3• 12 years ago

Holography is a specific thing, let's not confuse things by assigning magical powers to the Rift.

Holographic displays already exist, usually they're transparent, they work by letting you focus on things through the display and the display stays in focus, making things like AR and HUDs work much better than other types of displays.

acous4• 12 years ago

My mistake, but I'm not assigning magical powers. I'm aware of what a holographic display is, and the rift fulfils 3 of the 4 criteria. The lack of accommodation in my experience is not critical and the applicability to lytro is still valid.

win_ini• 12 years ago

I agree with most of the statements - but one aspect I think a lytro camera is better at: street photography.

Quick, shots of street moments. Shots where you have no time to focus properly, but want to focus the field of view on a subject...a lightfield camera is great for these situations and I think isn't played up enough.

NIL8• 12 years ago

The Lytro gallery:

https://pictures.lytro.com/lytro/albums/149429/embed?token=6...

I want this.

hnriot2• 12 years ago

a photo where nothing is in focus. great.

lytro needs a different name than photography because it's something else.

bobbles• 12 years ago

Are you serious? Click where ever you want to focus the shot

bridger• 12 years ago

I took a class that explained light field photography, but I thought understanding the concepts through static diagrams was difficult. I made a little Mac app that simulates a 2D scene of light sources, lenses, and light-field sensors. The sensors shows their captured output in a graph.

The source is at https://github.com/bridger/optic-workbench

A bit of explanation (and a link to the pre-built binary) is at http://www.bridgermaxwell.com/index.php/blog/optic-workbench....

(Sorry it is only for Macs. I would love to redo it in Javascript someday, once I know Javascript.)

spankalee• 12 years ago

I'm not sure that consumers don't care about bokeh, they just don't know how to achieve it. I've been using the Google camera app that does lens blur and everyone I show it to loves it. It's a little cumbersome to use though and require a still subject. Maybe light field could be one of the features that keep compact cameras around in the face of competition from phones.

I also wonder if there are computer vision applications. I don't know how accurate depth fields from stereoscopic images are, and maybe two cameras are not always practical...

dpark3• 12 years ago

Bokeh isn't just reduced depth of field. Bokeh is the quality of the out-of-focus areas. Shallow depth of field makes the bokeh more apparent, but the two are somewhat orthogonal.

Looking at Lytro's examples, e.g. [1], I would say that the Illum demonstrates poor quality bokeh as a result of the sensor limitations. The bokeh is quite noisy and pixelated (which is a strange thing to say about bokeh, but it's the result of the microlens sampling the Lytro sensor does).

The technology is very interesting, but looking at Lytro's pictures has convinced me that their camera will not suit my needs any time soon.

[1] https://pictures.lytro.com/lytro/albums/149429/embed?token=6...

erikpukinskis• 12 years ago

The thing that excites me most about Light Field Photography is that LF cameras are the only thing that could take a photo that would actually feel at home in an Oculus Rift. Binocular cameras fake the 3D experience by capturing two 2d images and fixing them to each eye. But in the Oculus you can move your head around so two 2d images aren't enough. You need an actual depth map in order to generate images in response to head movements. LF cameras can provide one.

http://www.cs.berkeley.edu/~ravir/lightfield_ICCV.pdf

nakedrobot2• 12 years ago

Not really. Without the correct IOD (interocular distance) [1] you won't get "correct" stereoscopy, and the effect you get will not feel right at best, and make you sick in the worst case.

A LF camera only gives a stereoscopic effect proportional to the size of the sensor. So unless you have a sensor even bigger than a medium format camera, you are not going to achieve the effect you are describing.

[1] https://en.wikipedia.org/wiki/Stereoscopy

mapt3• 12 years ago

The premise here would be two LF cameras, mounted rigidly in an IOD-like frame.

Or ideally, more than two, for flexibility of the IOD and a better stereo-vision fix, potentially used in 2D-3D compositing.

fuzzythinker• 12 years ago

I don't see the $1.5k Lytro is going to gain enough market share to be profitable too. The only option I see for Lytro is to miniaturize their technology so that it is significantly better than Google's software offering and to partner with a major phone manufacturer.

porker• 12 years ago

Could LFP be used to 3D-map a scene - assuming you can extract the data from their proprietary format? It would be great to be able to turn a collection of photos into a virtual world.

mapt3• 12 years ago

The problem with lightfield photography, other than the inherent image quality calculus of splitting a sensor into a bunch of lower-resolution, lower-illumination, differently-focused subfields, is that it's so easily simulated if anyone actually attempts to do scene capture properly - through adaptations of stereo-vision algorithms for higher-N arrays of semi-calibrated sensors, or near-infrared structured light (like the Kinect) for a 4D scene, or structure from motion for a static 3D scene.

The new HTC One M8 uses crap smartphone sensors, only two of them, without any structured light, with a first-generation stereo-blurring algorithm dumb enough to run quickly on a smartphone, and it produces a rough approximation of the same product as Lytro.

jychang• 12 years ago

As a programmer who also happens to be a photographer, that's flat out wrong.

The M8 produces a very crappy imitation of out of focus blur of what the bokeh of an camera with a thin depth of field looks like. [1] The linked article is actually being "neutral" and fairly generous; the pictures, frankly, look like shit.

It is difficult to properly do DoF, even if you know the Z-distance of every object in the scene. It's not just a simple blur of things that are deemed out of focus; things like spherical differences in the lens to produce hard/soft bokeh matter. [2]

"So easily simulated" couldn't be further from the truth. A bunch of academic papers have been written about this, [3] which include ways to reproduce effects that are similar to the "thin DoF" effect in large sensor cameras, but few try to tackle _all_ of the issues and nuances present.

[1] http://www.trustedreviews.com/opinions/htc-one-m8-camera-vs-... [2] http://jtra.cz/stuff/essays/bokeh/ [3] http://dl.acm.org/citation.cfm?id=2389316.2389318

dpark3• 12 years ago

To be fair, Bokeh on the Illum is pretty poor, too. The M8 bokeh looks fake. The Illum Bokeh looks noisy and pixelated (and the effective resolution looks far lower than 5MP to me). I wouldn't bother with either.

mapt3• 12 years ago

in [1]: "Now, this simulation above may look too artificial and unintuitive. Real lenses do not behave like this, right? They actually do. Just look at this picture below."

So he produces an accurate convolution model of how bokeh looks in soft/hard configurations, and that's proof it's not easily simulated? If it's not a straight Gaussian blur (as the HTC is likely using), that's fine - it can still be simulated.

What plenoptic cameras do is use a microlens array to perform focus bracketing. One can use a macrolens array as well, if one lives in a rich enough 3D-interpolating image processing pipeline.

Take a brick-sized lump of plastic, apply, say, 20 tiny cameras (each of four corners and center get a sub-array of medium_NIR_filtered-far-medium-near), and four temporally staggered Kinect structured light projectors on side, and you get all your cues double-checked by redundant information. Your algorithms get to play with direct low-resolution texture-focus info, stereo distance for a bunch of long-baseline pairs in two dimensions, redundant short-baseline pairs, active focus returns on laser dots as an anchor, stereo returns on laser dots, everything. One checkerboard calibration and such a system cross-calibrates itself and characterizes a position, focus, & lens distortion model for every lens.

Add additional lenses if you want to play with more of or some other type of bracketting.

The point is, attempting proper computer vision scene capture involves a crapload of sensors checking their results against each other, ideally diverse sensors and code. We simply haven't tried that, outside perhaps professional motion capture suites. The sky's the limit on such things.

[1] http://jtra.cz/stuff/essays/bokeh/#what_affects_bokeh_shape

Light Field Photography

Related Stories