But it wasn't a very good camera, and they completely missed the boat on making it a storytelling tool.
I took my Lytro with me when I walked with Team Torani (the Italian syrup people) in the Columbus Day parade that year, and did not get even one good shot. I wished I'd brought my little point-and-shoot instead, or just about any other camera, because I would have gotten dozens of good shots.
The pitch at the time was "you can put your photos on lytro.com, and people who view your gallery can click anywhere to refocus!" But who wants to do that? If you're looking at a gallery of photos, do you want to go clicking around to bring different things in and out of focus? I don't.
But for a photographer, this had some potential as a storytelling tool. Imagine having a photo that would do its own "pull focus" [1] when you view it, like in the movies. The photographer could first focus on one subject and then have it slowly change focus to something else to reveal something new to the person viewing the photo. Kind of a Ken Burns effect [2] but in the focus plane instead of - or perhaps in addition to - panning and zooming.
I remember one photo I got at the Menlo Park street fair that would have lent itself to this. I had the camera down around knee height, and a dog came up with his nose inches from the lens. In the background was his owner, an attractive young lady. It was a nice shot, and I imagined having it start with the dog's nose in focus and then gradually pulling the focus back to the owner.
Granted, this wasn't much of a story, but it was something. I couldn't even do that. All I could do on lytro.com was set the initial focus, and then leave it to any viewers to go clicking around to fiddle with the focus themselves. But nobody wants to do that.
It seemed like an opportunity lost, because they were so focused (pun intended) on the amazing technology and not on how to use it to tell stories.
To my mind their mistake was not having an open source library that could work with their capture format, including basic support for focusing and producing an output image for display.
Pretty much every new photography tech has had a community of amateurs and professionals spring up around it and (hopefully) innovate.
They missed out on that; all we got was the fun-for-a-few-minutes clicking around thing, which gets old pretty fast.
Oh, now you are making me wistful for what might have been.
If they'd had any kind of library or API, I could have played around with my Ken Burns idea by writing a few lines of code!
I've seen this same mistake over and over again going back to the earliest days of GUI apps: designing the user experience with hardcoded interactions instead of building the UI on top of a public API that enables others to create and experiment with their own interactions.
With careful and laborious use of Photoshop it’s possible to do most anything today, but with a light-field image there are a lot of creative controls that could be made easier, faster, and more reliable. (Of course, doing this properly would take hiring some signal processing gurus, color scientists, professional photo retouchers, and interface designers, and spending years on semi-open-ended research.)
The tech was really interesting to photographers and I absolutely think it could be adopted if built into a reasonable camera design. It’s very normal for photographers to carry many cameras and I knew people to carry Lyros some but I think some of it was that the company had two different users if was designing for, which isn’t usually a problem with cameras but this was so different and novelty that it mattered. I really think if they just made super nerdy products without trying to satisfy the cell phone crowd, they could have built a niche following in photographers.
JFK assassination researchers.
They absolutely tried hard, but I feel like they missed out on doing the one thing that might have really let the product take off: Releasing APIs (or at least protocol specs) for interacting with the camera over USB and wifi, and releasing a spec for their file format.
That might have given them a chance to really take off in the DIY and hobbyist market. After unboxing mine, I pretty much immediately wanted to use it to do the sorts of things that Google seems to want to do it. But without an easy way to dig into it, I lost interest pretty quickly.
Taking off my hacker hat and putting on my consumer hat, it just wasn't going to take off. My cell phone is even more portable, takes higher resolution images, and makes sharing them much easier. Getting to refocus after the fact wasn't that great a trick, because my cell phone camera already does such a good job at autofocusing.
Taking that one off and putting on my hobby photographer hat, same thing. Decades of taking pictures has left me with enough of a sense for where I want to put the focal point and what I want to do with the bokeh that I'd just as soon do it when I'm taking the picture. There was some initial fun of playing with a new toy, but ultimately I realized that doing it after the fact took what has historically been one of my favorite parts of photography and turned it into just another editing chore. That, and, even when I'm not 100% sure what I want to do with the focus at the time I'm taking the photo, my mirrorless's burst mode can accomplish essentially the same thing.
But if I could have strapped that sucker onto a robot and played with using it for 3D scene reconstruction or something. . . that would have been fun.
That's where I find myself always hating autofocus (even on a fancy camera with focus tracking and motion detection and such).
My problem is more that with animals, it's hard to make sure it's picked up the head vs the shoulders, say, and with shallow DOF that can ruin the shot. But then, I don't have eye detection, so maybe that would do the trick. :)
I'd assume the Illum (which I've never had a chance to play with) would be way better on that front.
That tech in dash-cam video is priceless
With dash-cam video you can just chose a fixed hyperfocal distance and have everything in good focus.
You certainly don't need creative DoP and bokeh...
You can also see past occlusions, something not possible with stereo.
With a larger and more distributed camera array, cameras could be pointed everywhere so that every point in the field has a certain level of coverage.
Using stereoscopic as an example, you can synthesize a 2d image with the center being between the two sensors. The further apart you place the sensors, up to some limit, the more you can "rotate" the scene. But you lose a lot of data when the viewing angles become too extreme. A light field image can be used for the same effect, but because it's one sensor your ability to rotate the scene will be constrained (as if you had two sensors very close together).
Apparently the examples I saw years ago on Lytro's site aren't available anymore. It was a neat effect with their initial camera, I imagine higher quality cameras could do much more.
A neat thing with light field sensors versus stereoscopic is that you have far more freedom in how you move around the scene. With two sensors you can only move "left and right" or "up and down". With light field data and one sensor you can move in any direction.
I think it worked by making a point cloud that fits the camera observations.
Obstruction is not a big issue in that use case, but if there was obstruction, the system could choose to ignore pixels that were obstructed when constructing the image.
(I spent a lot of time reading about light field stuffs back when Lytro was first announced, I was also tangentially connected to a synthetic aperture radar project so it piqued a lot of my curiosity at the time. I haven't kept up with the state-of-the-art since then, though.)
Light field sensors could greatly aid self-driving vehicles as a significant improvement over standard cameras and stereoscopic configurations for object detection. It's not as good as Lidar in some ways, but it's much less expensive and offers some of the same kinds of data (which means greater portability of algorithms or techniques between the two sensor types). And since cars need to see the same visual cues humans do, we need cameras anyways (for reading signs and road directions, lane detection, etc.) so using a light field sensor could offer a twofer. Improved object detection plus the ability to do visual detection we already need.
segmentation of foreground/background, becomes much simpler.