First of all, you need a lot higher refresh rates, because you tie in your ocular-vestibular system which is very sensitive, compared to the visual feedback required to be comfortable sitting in front of a monitor. It's still unclear where the upper limit is, but the lower is probably at least 120 Hz. The consumer devices being released soon are running at 90 Hz, and that's the bare minimum. I've heard "1000 Hz" coming out of Michael Abrash's mouth, but that was somewhat of a hypothetical number.

Secondly, higher resolution. You'll want at least 4K in the near future, because with a lower resolution display that close to your eyes, you'll discern individual pixels and the actual space between them (causing the infamous "screen door effect").

Even further, strict latency requirements on an already-busy system prevent you from doing any interesting compression.

My experience is with Occulus. Full HD is not enough as you clearly see the pixels. I was so distracted by the "low" resolution that I couldn't immerse myself at all.

Probably 4K will be better, but I suspect it still won't be enough

Those Codecs that compress video well have considerable latency. Raw video out has been the single signal with the highest bandwidth in almost every computer.