You're right! That's absolutely possible. I've had the experience of working with such offices. I've also had the experience of working with a partially distributed team that encourages working from home. In each case, I found distinct disadvantages compared physical co-location. Both for companies and for employees.

I share your hope for the possibilities offered by VR, but I'm hesitant to shape urban planning policy for decades around it.