If you only hire within the pool of people who are willing and able to commute to your office, you'll both need to pay more and have less choice. Workers are more tired as they spend hours commuting every day.
On the other hand, it's harder to develop juniors if all your seniors are working from home. Written communication lacks context that helps difficult conversations move forward. Silos build up as people in different teams never see each other.
In the long-term, I believe the broader benefits to society and the planet mean fully remote working will become almost universal. In the meantime, a hybrid model of mostly remote with broadly-attended meetups is the best combination. But some companies will decide that everybody being in the office all the time is right for them, and they can perhaps leverage some benefits of doing so.
How often do you feel is enough?
Assuming the "typical" company made up mostly of people who've been there for more than a year, and where commuting is easy, maybe every 3 months is enough. New teams (even within such a company) should aim to meet weekly while they're in the forming stage. Small companies made up of very experienced ICs perhaps never need to meet at all.
I guess the best way of working it out is to ask people, and review what works.
I really can't see it being worth it in massive, overpriced cities.