The model for the interaction has already been figured out by all of these tools that people are trying to create, hasn't it? Point a probe into the world and get back a signal that corresponds to some geometric property of the world - distance from the probe, edge-detection, etc. It's like virtual reality: replace the world with a rendered scene and the lidar range detecter with something like a Vive wand.