I don’t think I agree with him, but that doesn’t make it “not a reason”.
Reason I disagree with him is that if workplaces rearrange themselves to not need physical presence, all the jobs suddenly get done by whichever genuinely third world nation can put high bandwidth internet connection into literal shantytown slums the fastest.
If this was a gradual and planned-for change, starting with boosting the salary expectations of the poorest in the world up to our level, that’s one thing. Doing it to solve housing cost differences between rural and urban areas in developed nations will just blow up in everyone’s faces with unhandled side-effects.
There are many natural reasons why being physically in an office is advantageous. I can have face-to-face meetings with multiple people without conference call issues, I'm available to go visit clients local to the office, interactions with colleagues are just easier due to proximity, I hear the office cross-talk that is unavailable when you're remote, I form closer personal relationships with my colleagues because of countless unplanned interactions, I can perform physical tasks like installing servers, etc.
Granted, there are disadvantages to working locally too - but the complex mix of advantages and disadvantages feed into the decisions that everyone makes in how they run their businesses, where they choose to be employed, etc.
Does it not concern you that you're advocating overriding all of that decision making with some one-size-fits-all government policy that would criminalize people simply working the way they wish to?