The real problem tends to be when you have a few outliers. Say, a 10+ person team where everyone is next to each other and are all in the office most days except for one or two who are fully remote. That can be a problem. It can also be made to work. But everyone needs to work at it and be committed to it.

To be honest, the situations where I have personal experience are mostly more senior teams where everyone is doing a lot of travel anyway, a decent percentage are fully remote and the rest are often only in the office a few days a week. And everyone is doing as much communication (if not more) with other teams and people outside the organization as they are internally anyway.