If I were building a new business, I would use both cloud and colo. But I do understand that everyone don't have that luxury.
"This is the cost of scaling, this is the cost of owning our own infra, how does that fit into our budgeting and requirements?"
There are plenty of tools and systems that can present a sufficiently linear cost relationship to load and usage that, should your COGS versus revenue make sense, the marginal cost of increased cloud resources a no-brainer--especially versus always-paid-for hardware. If you don't have such a linear relationship you're as much in the position of deciding whether the project is viable as you are anything else.
The lead time for Hetzner or OVH is measured in minutes and is appropriate for the majority of use cases. Old-school providers like these used to run the entire internet before the cloud craze started.
Colocation & own datacenter is not a good choice for most startups. Seems like a lot of people here miss a step and only go from one extreme to the other. There's a middle-ground of renting bare-metal that gives you amazing price/performance with none of the drawbacks of colocation or running your own DC.
Also, the cloud offers managed software as a service. You don't have to manage your own HA DB cluster or PubSub. It's all just there and it works. That can save you a lot on technical labor costs.
But yes, I do agree with your point. If you don't know what you're doing, you can nuke your budget super quick.