If the cloud to bare metal price difference is 10x.. you could easily just buy bare metal = 2x your peak load, and still come out ahead..
And that's why I come to HN, lobsters, etc :) so that I can read and learn from others' experiences...
Google really does need a service that can scale to 100M users.
I doubt someone starting a SaaS to make garbage truck software needs to. Or at a minimum, at the time they need that kind of scale, they will have plenty of money to throw at hardware. It's amazing how many users a single maxed DB can handle.
If you're trying to reduce costs on a complicated usage pattern, then you'll probably want to have someone on staff or consultant who knows how to deal with those things.
I don't know your experience, but bare metal has at least an order of magnitude greater efficiency at one or two magnitudes lower cost. If you want to pay $1MM/mo for something that could also be done for (generously) $20K/mo (plus a couplefew salaries that add up to much less than $1MM/mo), that's your business, but I don't see how the use-case you describe requires cloud/AWS.
The thing that people don't get (or have forgotten, or...) is that you don't have to spin up hundreds of instances in the way you do on AWS, you just handle them on a machine or three that already has enough headroom to handle them.
(all hypothetical blather shaped to support my perspective and disprove the utility of yours, of course ;)
Not so sure about most people learning this...though, I will say I'm probably a late bloomer compared to most on HN :)
What you state makes sense. I think my tone was dismissive of the idea that running your own machines is worthwhile. To add to that, I haven't (yet) worked on large-scale systems so my comment is purely speculation.
If you look at what it would cost to run bare metal with top shelf companies, it's often as cheap or cheaper than spot/preemptible instances at the cloud shops.
It was a system that had fixed scale, high licencing costs and did not bridge well into software that increasingly leveraging the AWS environment. Egress was least of our worries relative to integration complexity in services ecosystem.
You have to be a very very large scale before self managed services make sense; as you suffer at integration layers for every developer that is even an inch off the well beaten path.
I can see how running game servers—with a (somewhat, maybe?) predictable load of players—can be done efficiently on baremetal and with colocating.
For another business that has huge peaks of demand, such as analytics with dynamic queries, I fail to see how baremetal can compare to spinning up hundreds of instances on-demand.
Perhaps it comes down to what services you offer your clients and how you implement them?