It's likely for VR, the 'reverse' of the Lytro allows projection to the eye at multiple focal points.

I don't know about "likely", but that is an awesome idea.

This already exists: http://lightfield-forum.com/light-field-camera-prototypes/nv... (among others)

Wow, that looks awesome! For something at the demonstration-prototype stage in 2013 I'd have hoped they would have something at least in development by now.

They have a fully working version of it: https://vr.lytro.com/within/hallelujah

Looks awesome, but only an "experience" rather than something you can buy? I guess it would need an ultra-beefy system to run it, and is probably extremely expensive and not supported by much software (yet).

They run it for you - it is a capture service. https://www.lytro.com/immerge#immergeDetails

We're talking about two different things. Hallelujah is interesting because it does light field videography, but what I linked was a light field display.

My top candidate for AR in the longer term is a scanning pico-watt laser which projects the image directly onto your retina.

Why would it be priceless?

With dash-cam video you can just chose a fixed hyperfocal distance and have everything in good focus.

You certainly don't need creative DoP and bokeh...

Just spit-balling, but could it be used for cheap depth perception, like a LIDAR alternative?

Probably, but so can two normal cameras, like the ones humans have in our heads.

stereo cameras only give two viewpoints, it requires a lot of processing power to re-create the 3d scene. a lightfield gives you much more information making reconstruction much simpler.

You can also see past occlusions, something not possible with stereo.

How can you see past occlusions?

A stereo camera array can see past occlusions in the sense that some things are occluded to one camera but not to another.

With a larger and more distributed camera array, cameras could be pointed everywhere so that every point in the field has a certain level of coverage.

The GP said you can see past occlusions with a Lytro and not with stereo, and that sounds exactly the other way around to me (as to you).

You can do it with both, actually. Light field data allows you to simulate different viewing angles of the same scene. Depending on the sensor quality you can use this to "see around" objects which occlude the scene from a specific angle (like if you view the image from the center of the field of view). Some limitations, of course, similar to stereoscopic imagery.

Using stereoscopic as an example, you can synthesize a 2d image with the center being between the two sensors. The further apart you place the sensors, up to some limit, the more you can "rotate" the scene. But you lose a lot of data when the viewing angles become too extreme. A light field image can be used for the same effect, but because it's one sensor your ability to rotate the scene will be constrained (as if you had two sensors very close together).

Apparently the examples I saw years ago on Lytro's site aren't available anymore. It was a neat effect with their initial camera, I imagine higher quality cameras could do much more.

A neat thing with light field sensors versus stereoscopic is that you have far more freedom in how you move around the scene. With two sensors you can only move "left and right" or "up and down". With light field data and one sensor you can move in any direction.

Ah, I see what you mean. You're constrained to viewpoints on the sensor, though, no? So usually a small square.

A few Superbowls ago Intel demoed a supercomputer which could take in live video streams from hundreds of cameras, run a batch job for about 30 seconds, and then be able to synthesize video from arbitrary viewpoints above the crowd.

I think it worked by making a point cloud that fits the camera observations.

Obstruction is not a big issue in that use case, but if there was obstruction, the system could choose to ignore pixels that were obstructed when constructing the image.

Right. And with the initial cameras, which were neat but not breathtaking in their capabilities, you really only got a small ability to move around the scene. A larger sensor or a pair of these would offer greater capabilities for this application. I believe using an array of sensors was how their video camera worked, but I'm not 100%.

(I spent a lot of time reading about light field stuffs back when Lytro was first announced, I was also tangentially connected to a synthetic aperture radar project so it piqued a lot of my curiosity at the time. I haven't kept up with the state-of-the-art since then, though.)

Sounds plausible to me.

Do you really want “cheap” depth perception for a self driving car?

Perhaps read that as "inexpensive" rather than "cheap" if you take cheap to imply low quality.

Light field sensors could greatly aid self-driving vehicles as a significant improvement over standard cameras and stereoscopic configurations for object detection. It's not as good as Lidar in some ways, but it's much less expensive and offers some of the same kinds of data (which means greater portability of algorithms or techniques between the two sensor types). And since cars need to see the same visual cues humans do, we need cameras anyways (for reading signs and road directions, lane detection, etc.) so using a light field sensor could offer a twofer. Improved object detection plus the ability to do visual detection we already need.

Yes? Why would you not want it to be cheap? Current self driving LIDAR units costing $75,000 seems like a pretty major barrier to mass producing the things.

Yeah dash cams are super wide angle. I don't see the point since everything will always be in focus.

The reason I was initially thinking of was that a dash cam with multi focal distances could provide a way to see a dash vid from multi perspectives - and basically make a single dash cam “seem” like many individual cams where you can get the focal clarity of distance from more than one perspective? Else, my understanding of the lytro tech is false.

A light field allows you to generate dense 3d point clouds, even when static.

segmentation of foreground/background, becomes much simpler.

I think its more that what the tech is for the Pixel 2 Super XL edition or Pixel 3. That would let you build a 3d photo and post it to Google Photos. Or maybe they could partner up with a 3D printing company the way you can print photo albums.