Not sure that's universally true. We have probably tens of thousands of EC2 instances in our infra right now. Just the cost alone to build out on-prem infra and migrate things over would likely wipe out a half-decade or more of cost savings. And in the meantime our shift in focus would hurt our market position.

If we'd started in the beginning (11 or 12 years ago) with our own infra, I imagine our recurring infra costs would be lower right now, but I also suspect that our 3-person founding team would have failed to produce an MVP before their initial time and money ran out.

If I were starting a company now, I'd probably do things in a more "platform agnostic" way such that a cloud->on-prem migration might be easier. But I still never expect it'd be easy.