I keep thinking there will be a significant reduction in complexity there. The intensity of a pixel in a hologram is essentially an integral over all surfaces visible from that point. So imagine a rather complex formula applied for each surface for each pixel. Then imagine holographic bounding boxes that compress complex geometry into a few holograms of what's inside. This would reduce the n^4 back down, but the resolution required for holograms is still very very high. But we could use fancy GPUs to evaluate the integrals.
Just hand-waving thinking here...
But compressing a light field and projecting one into an eye are totally different things. Your display needs to be capable of displaying all 3n^4 possible intensities at different times. Depending on the scene and where you're looking, you can get away with showing only a smaller handful of those (some megapixels) but the display still needs to be capable of displaying them all.
If your display is just fixed LEDs behind a microlens array then you still need n^4 resolution. Like, a megapixel per pixel. Most of the pixels can be off at any given time, but you'll need them all to display arbitrary scenes.
Otherwise, the existence of those additional rays is what allows for focus accommodation: as you focus in different planes, it 'shuffles' the light around, to create sharp edges (where similar light rays line up) or blurry foreground/background (similar light rays strike different portions of the retina -- and if any are missing, there is a 'hole' in the blur disk).
Pedantic, perhaps... but these are the stakes.
I think Google is on the right path with Tango-first. Didn't used to think so.
Items in infinity emit parallel rays, items very close to the eye emit rays with a large spread...
Those four dimensions are projected into two on the retina, but the exact projection is a function of where the person is focusing. (That's how we understand focus; by changing the lens parameters, we can focus at different depths in the scene.)
So three dimensions: two for position and one for "amount of off-axisness".
Full light field displays have to draw every ray, not just every location; that's two additional dimensions. Resolutions scale badly, and if you make any of the four dimensions lower resolution, you'll get ugly artifacts.
Microlenses aren't the limitation. Pixel density is, both in physical manufacturing side and when refresh / drawing.