This feels like the similar kind of problems of understanding how to accommodate the human into the platform. Early mobile platforms were just "desktop, but smaller". You were expected to interact with the device using a stylus. Small checkboxes and UI elements were common. The device needed to regularly tether to a desktop.
Lots of VR games still allow you to move/rotate using thumbsticks. Part of it is that these games are ports of desktop experiences that wouldn't work any other way. Over time developers will better understand what works and what doesn't, and will design games/apps that don't make you nauseous.
There's also likely a bunch of vision problems that still need to be worked through. I have no expertise in VR but with my limited usage it feels quite close, and I think these problems are not insurmountable.
I'd argue that there are many, many puzzle pieces still needed to begin imagining an AR future. While the Newton showed an incomplete vision of its future, it begged all the right questions. My kids Occulus Quest, on the other hand, just makes me feel motion sick...