Sounds like you dodged a bullet there.
Hire good people, entrust them with executing on core business needs, allow them to choose however they make the magic happen.
With enterprise I'm playing more with averages so that one person on the team that may not be carrying their weight isn't going to drag down the entire business. For a startup where it's more critical that everybody be firing on all cylinders it makes sense they want to have more control and accountability to make sure their limited resources are being spent appropriately.
In person, it's trivial to see everyone coming up to my desk, hear me jumping into team discussions, and so on. Remote, it's on me to let management know that's what my time is being spent on, and if I'm lying it's significantly harder to verify.
My recent anecdote: Startup out of stealth a month ago informed me during 2nd round that they were only considering programmers willing to relocate and live in Austin. No remote option whatsoever. Asking what would happen if there are additional lockdown measures reinstituted, for Delta or even just whatever other 'virus-of-the-year' comes along, they didn't have any answer other than to repeat Austin-based only.
Bonus: Their founders and early execs are today only located in SF bay area and have no plans to relocate and the in-office only policy would only be for the devs.