You might be considering a lot broader things to be spatial computing that are short of the sense being considered here.
Yes, representing objects and annotating trivial scenes is interesting and is starting to happen today.
But free-form automated representation of the environment in ways that the spatial shortcuts we rely upon ("inside", "on top of", etc) can be used, (non-graph) paths of different types can be solved, and actuator kinematics can find and interact with things, etc: it's really hard.
AR and VR and being able to interact with object meshes and geometry is a start. But computational geometry and encoding world constraints in a useful way is really hard. We'll snatch some incremental parts from AR and vision research, but we'll need some breakthroughs in the space, too, to do the really cool stuff.
At its core, spatial computing is about queryable reality. 3D scanning of objects, AR/VR, etc are all adjuncts to this. They are ways of consuming the data model or synthesizing non-real parts, not creating the base model of reality. The latter is the hard part, the rest doesn’t matter in the big picture.
What you describe is a broad vision that may or may not happen at some point in the future. I agree about the queryable reality part but disagree that AR/VR, 3D scanning etc. "don't matter". Because in a lot of real-life use-cases you don't need the "real-time 3D mirror" but instead can get 80% of the value with something 10% as complex.
Adoption of spatial computing will not be binary "once someone pulls it off". It's going to be gradually, step after step.