That, plus the collaboration features would make HoloLens pretty awesome for any kind of collaborative product design. Take cars, for example. Right now, car mockups are done in clay, and while clay is easier to mold than metal, it still takes time and limits your ability to iterate. But with Hololens, you can have a design review where each of the participants puts on an AR unit, and now you can all see the same car and make changes to it in real time.
I'm excited to see what sorts of applications will be opened up by being able to display virtual objects in a "real" setting. I also agree that this won't be consumer-facing technology (at first). This is the sort of thing that'll take off with architects and engineers before it takes off with "normal people".
The depth maps can get dynamically updated, so it's not insurmountable, but currently the process seems to take a few seconds. I'm not sure if this is a design decision, a processing limitation, a hardware data capture limitation, or something else entirely.
Pure speculation, but I would bet that it generates the room geometry at startup and then doesn't change it later. The device has limited processing power and needs to hit a high frame rate, so doing that makes sense.
A person walking into the scene wouldn't be accounted for in that case. It's also dependent on scans of the environment and the quality of geometry reconstruction, so things like a gap between objects that you didn't see from a good angle could easily run into that issue.
Another interesting depth issue is when it ran to the back of the table and straight on to the floor as if they were continuous, because the view angle made it so the table and floor were right next to each other only separated by a line, so the fox didn't do a jump animation. I suspect a depth map of the room needs to be constructed to solve that issue, no mean feat.
These kind of issues just show me this tech is still years away from being what we'd expect.