"Probably" ? Sounds like you don't actually have any experience on the matter and are guessing.

There are numerous fully-remote companies that make it work. That doesn't mean it works for everyone, but it definitely proves that it isn't inherently "excessively limiting".