I have worked both remotely and in-office. In-office is good for collaborating on how to architect code, in the early days of a product. But as the organization matures, you have an established architecture, style, etc., and are adding or tweaking features. When you get to that point, remote work is excellent. You are able to slice off work to people who can sit in their corners and execute on those smaller pieces of the vision. They are not interrupted by others, they can both focus and relax as works best for them. It works. (And, FWIW, when we did major architecture work, we did fly into a shared location for a few days to hash things out together.)

That being said, people are all different - specific people may work better in one environment or the other. You may just personally fit in well in an office.