What's much more interesting to me are exploration of innovative interfaces. For example, with full-body tracking you could map body parts to different synth parameters or algorithm parameters and then allow the user to make music by dancing.
The sky is really the limit in terms of what body/head/arm tracking and VR can allow you to do with innovative interface design.
For something as refined as the sensation of twisting a knob, however, I don't think that the current VR controllers are enough. Maybe once VR gains a bit more mainstream adoption, we will get VR glove controllers with a bit more complex haptic feedback mechanism, which should allow for the "twisting the knob" sensation to be implemented, and then it will be a total gamechanger. I would looove to have my DAW and synths in VR, as the spaciality of VR is perfect for this kind of workflow, but we gotta wait a bit for tech to catch up.
I think current gen controllers could work well enough here to be useful. The modules might have to be oversized, and you’d need to have a visual indicator that “this is the knob you’ll grab if you pinch now,” with good visual position feedback. But when I play Beat Saber, part of the fun is how my every slight twist is reflected exactly by a freaking lightsaber! The fine motor control is already present.
It’d take some getting used to, but still miles ahead of mouse/trackpad. The nice thing about VCV being open source is...someone can just build it, and it can be iterated upon as the haptics improve.
There's the Hot Hand[2]; the bassist is using one in [3]. There's still a lot more that could be done.
[1] "The V Motion Project" https://vimeo.com/45417241
I’m interested to see what the availability of cheap VR does here. With today’s room scale VR, using knobs and sliders in a 3D space is feasible. Being able to develop spatial memory, even without tactile feedback, makes for a compelling alternative to using a mouse and screen. Once VR/AR gets tactile feedback like this glove[0], it gets even closer to parity with hardware.
If the module controls in VCV Rack are standard components (button, knob, slider, etc.) then it should be pretty simple to translate them into 3D controls. Though I guess buttons may not work as well as switches, I haven’t encountered them in VR yet. Anyway, with those component mappings in place, any VCV module should be displayable in a virtual rack with no extra effort—it’s just a procedural UI change, generated from the existing 2D layout, that doesn’t impact the audio engine at all.
[0] https://youtu.be/OK2y4Z5IkZ0