Generally speaking, most sysadmins at small places do a horrible job engineering for failure. Or, worse, they don't have professional SA's, just devs who know enough to be dangerous.
So while you don't run into a US-East EBS failure every now and again that affects millions of servers, you are subject to random unforseen failures because somebody didn't do X correctly. (Where X could include: disk configuration, HA configuration, DR plan, backup/restore plan, os management, firewall management, etc)
There's no one right answer here. I've rolled specific hardware solutions managed by a single team for certain applications, and everything was fine. I've been stuck working within the bounds of a managed services provider, and things worked too. Cloud is just another model.
EC2 lets you roll a globally distributed solution with good tooling for low cost.
This is the Crux. There are other options available, but going cloud makes it very easy to roll out something big without the necessity to have experts in all the lower level parts of your infrastructure. It lets you focus and the price for that is absolutely reasonable
I think engineering for 'The Cloud' can be just as time consuming if not more so for start-ups. For Amazon at least you have to engineer for failure. With a dedicated server configuration you still do need to engineer for that, but you aren't worrying about if depending on the solution providers block storage is going to take certain aspects of your architecture offline when it goes down (yet again), or if the portal won't allow you to bring up/down new instances during a very critical period of time, etc.