Have you tried the Hololens? It's exactly what you describe in the last sentence.

Exactly. The quality and FOV of the original hololens is lacking, but the latency and tracking are mind blowing.

It had some rather jarring failures when I tried it in an office (lots of high-contrast edges everywhere, you'd think it'd be near-ideal...), but overall yea - quite good. When it works it's remarkably stable.

And agreed on the FOV and image quality - it's pretty terrible. And it doesn't help that the press around it has been incredibly misleading about it. That, and the abysmal gesture-detection have broadly left me at the "neat tech demo, why did you build more than a dozen or so?" level.

I wonder if it would be possible to move some processing into the display. It's not the same, but in particle physics pixel detectors, which are basically CCD sensors, people have been working to move the readout electronics into the pixels. There, the motivation is radiation hardness and cheaper production. Here, it could be decreased latency. What if you could put a super small shader in each pixel, and do a final processing step like 'timewarp' right there? Or a part of your graphics memory is directly your display, no bus in between?

That being said, I tried a couple of VR solutions and I don't really have a problem with latency. What I find more irritating is that I cannot focus well on different planes. You do have the effect when you cross your eyes that you put images from different planes together - although badly, because even if you don't see the pixels, you do get some kind of moire pattern when shifting planes (I can't describe it better). And what you don't have at all is depth blur. I think both can only be solved by true light field displays (if that is even possible). I hope there will be a breakthrough with holographic projection at some time.

> What I find more irritating is that I cannot focus well on different planes.

You might be interested in this Oculus VR research then:

https://www.oculus.com/blog/oculus-research-to-present-focal...

AFAIK all these VR/AR devices implement a workaround for that.

Specifically, they render stuff to a texture larger than the displays. When that’s done, they cut a rectangle from that texture to account for the head’s rotation since when the rendering was started. If the head didn’t rotate, they show the rectangle from the center of the texture. Otherwise they show off-center and/or rotated portion of that texture.

While it would be nice to not cheat this way and truly improve rendering latency not just head rotation latency, it’s overwhelmingly harder to do so, at least on current generation GPUs.