I want 3D emacs.
If someone gets the right metaphor going there, I think it could work great, but the naive solution of 1:N direct scaling the position of your hand to a co-ordinate in space isn't it. I imagine it'll be something like direct correlation to get 'in the neighbourhood' and then an easy modal toggle to a 'precision' mode, where you move something in small increments along just 1 or 2 dimensions, using a comfortable binary gesture.
Or maybe AI will get really good at guessing what alignment the user wants.
Right. The Autodesk people figured that out by 1990. It's like trying to draw while wearing mittens. And they had fingered gloves as input devices. During the 1990s, there were many attempts at tactile interfaces. None of them ever caught on. My favorite was the levitated metal ball. It rested over a shallow bowl, and offered all 6 degrees of freedom, plus it could resist your movement and push back.
VR is good for looking at 3D models, but not for working on them.
Not really. I tried Jaron Lainer's original VR rig back in the 1980s. The Autodesk system from the 1980s. The W Industries system ... the HTC Vibe. The Microsoft HoloLens. There's been progress, but usability really hasn't improved all that much.
You can shoot at people. That works fine. You can sort of use swords. Everything else, not so much. That's why Beat Saber is still the most successful VR game.
Some Autodesk people once thought that VR would make it easier to use CAD, because selection in 3D with a mouse is such a pain. Didn't work out, and 3D selection with a mouse got much better.
If anybody gets this right, it might be NVidia Omniverse. That's an attempt to do for 3D what shared text editors do for text documents.