>you get this weird dance of finding a spot sufficiently far away from each other to be able to stick with the policy and not also hear each other directly, with no latency.

See if you can talk to your boss and get everyone over-ear headsets with a boom mic, and spend some time tuning them so the mics only pickup the person talking.

That way you can have several people all in the same area all on the same or different calls, and for the most part it won't be an issue.