...well the scope of each team for which this "all-on-site or all-remote" principle holds is debatable (i.e. if we're speaking of a project team, department, set of people working on a given product or module within a system, etc). But the principle, I think, is still a valid one.
The real problem tends to be when you have a few outliers. Say, a 10+ person team where everyone is next to each other and are all in the office most days except for one or two who are fully remote. That can be a problem. It can also be made to work. But everyone needs to work at it and be committed to it.
To be honest, the situations where I have personal experience are mostly more senior teams where everyone is doing a lot of travel anyway, a decent percentage are fully remote and the rest are often only in the office a few days a week. And everyone is doing as much communication (if not more) with other teams and people outside the organization as they are internally anyway.
The thing is that, in many cases, there is no "all-onsite" especially for larger organizations.
Your field people (sales, system engineers, etc.) are going to be wherever your customers are.
Acquire a company in a different city? If you force everyone to move, you're probably going to lose a lot of the value of that company?
Work with open source software? Many of the upstream developers are going to be scattered all over the world in any case.
And as soon as you're in multiple locations even within a given city, you're really not much different from being in two separate cities at that point.
Certainly, there are challenges. But I'm in a very mixed environment, which I think generally works pretty well.