It's a real myth that the cloud magically saves me a sysadmin. I have found almost exactly the opposite. Using the cloud effectively takes more time and more expertise. Debugging the cloud effectively takes WAY more expertise. Combine this with the fact that someone has to be able to architect a system to fit within the constraints of being on the cloud, and you're down extra employees.
The difference is in: "Eh, it's been down for 3 days? Sigh. Just reboot it." vs. "Um, that request hung for 93 seconds. Why?"
In the first case, the cloud is fine.
In the second case, someone is going to have to traipse over an enormous amount of systems (which you don't own and can't always instrument) and variables (some of which you aren't even aware of existing) to hunt it down. If, however, you can say "Pull that off the cloud into our own systems and keep an eye on it." you have made your debugging life a lot easier.
Of course, once you have the ability to do that, your team realizes and starts asking: "Given how much time we spend debugging issues with the cloud that aren't actually our fault, why are we on the cloud again?"
I always smile when that realization kicks in. Now I generally have to stop the team from pulling everything off the cloud. However, that's a much easier task.
For the hardware side of things every colo I've seen has a "remote hands" service.
AWS is also about 2749 times faster and more efficient, as measured by me the last time we ordered hardware on AWS and dell simultaneously.
Bandwidth is almost always cheaper at a colo. Most compute instances are cheaper to buy and rack if you have continuous loads. Disk ... is tricky.
It is faster and has lower up front costs to get clouds up and running initially. For most startups who are going to fail, that's Good Enough(tm).
If, however, you continue existing for a while, the other things start to add up at much lower levels that you would expect. I'd say the crossover is around when you are spending about $15,000 per year. Your colo is about $10,000 of that per year and you can rack 5 new machines every year for the remaining $5K. That's not that much for an actual business.
Cloud is good for your initial startup and for bursty situations. Once you have continuous loads, you need to be moving to pulling stuff back to your own hardware.