We did have quick turnaround - we were running everything on Xen VMs on LLVM internally, it was trivial to throw up new instances, snapshot them, etc. Our demands weren’t changing that quickly, so there wasn’t that much need to bring on new hardware all the time. A small number of dual socket machines can go a very long way these days. We didn’t have the same kind of multi-DC redundancy that AWS can give you, no, but over those years, many AWS based services were down multiple times due to being focused in Virginia, so it’s not an automatic win for AWS. The amount of time we would have spent getting that running and maintaining automatic multi-zone failover on AWS likely would have swamped the benefits for us, though, and I think are usually overkill for a small company. YMMV.
I was a cloud skeptic and ran Tech Ops (including our DCs) for years. About 5 years ago, it dawned on me that even owning the whole budget for Tech Ops, that I wasn’t capturing the full costs of trapping my org onto our in-house solutions.
At tiny, small, and medium scale, cloud is obviously the way to go, IMO. At large and huge scale, I think letting some hybrid leak in where systems change rarely and cloud costs are WAY out of line (DropBox storage, Netflix CDN, etc) makes sense.