Going in this direction basically takes the level of resolution and precision in image reproduction we've achieved back a decade or 5, since the "pixels" or resolution units are spread over many more views than just 1 or 2, or perhaps something more recognisable as a continuum of "infinite" views, or whatever number is visually indistinguishable from infinity.
In these kinds of displays, you have a light generator (LED or flouro or reflection) with the same surface area as a normal print or display. But to produce the directional light, for say the left eye, you must block the portions of the image that relate to the right eye from the light travelling to the left eye. And vice versa. This halves the amount of light for 2 directional images. thirds it if you want 3. So you end up with an image that is much darker then normal. you must compensate by generating 2 or 3 times the amount of light. This problem gets worse the more "views" you add on. So if you want 10 omni directional views you have to generate enough light for all of them, since, even if you set everything to "white", as you suggest, most of that white is getting blocked from your view.
Or to put another way, you get allocated a smaller source 2d image plane surface area to generate your viewpoint.
http://www.exploratorium.edu/visit/central-gallery/giant-mir...
The kind of system I'm talking about needs a 2D image with enough "pixels" to fill a volume convincingly instead of just a 2D plane, and the optics would need to be far more complicated, precise, and created at a very small scale which would appear as a textured surface like a fresnel lense or one of those 3D lenticular stickers you sometimes see. Then once you've figured that out, you need to get the projection and the optics to line up precisely- unless you can figure out a way to build in some tolerance to the alignment of the projection.
Aside from that, the goal is essentially the same. To produce a field of light coming out of some "window" with the same directional qualities as the light coming out of a real window.
I don't know how cumbersome the lenses would have to be, or how insanely fast the system would need to operate, but it's at least possible in theory.
that's fine for a display but is obviously impractical for a large scale theater projection system. Just an arbitrary possibility I thought of just now: suppose the projection screen had a retroreflective surface- That is, light projected at the screen gets returned at exactly the angle it arrived at. Combine this with a domed mirror and a backwards pointing projector or set of projectors, with all the requisite optics math and geometry work, it may just be possible to project a lightfield at a screen that bounces back at the audience and appears as a tangible hologram to them.