I think this is the important part of the debate. Many people don't use these features, but with EC2 you are paying for them regardless. For example, you pay for the ability to pay by the hour (in terms of increased costs), even if you never plan on doing so. Yes, reserved instances can help, but they don't fully cover the cost difference.
So it really does come down to what features you need. If you do use all the features, I'd imagine that EC2 is a great deal, because everyone else is subsidizing your use of them. If you don't want any of the features, well, then you're subsidizing everyone else.
I think the "cost shouldn't matter" argument isn't relevant. If cost doesn't matter, it is hard to make a case for EC2 vs spending the money to develop your own infrastructure + hiring lots of people + vastly overprovisioning your own datacenters.
"the constraints EC2 imposes on you forces you to build fault tolerant, shared-nothing systems."
EC2 forces you to care immediately, not later when the shit hits the fan (which it always does, eventually).
S3, SDB, CDS, etc.. Yes, those make things easier. EC2 Does not.
Is this from experience? If so, please share it.
The way I look at it is if all these services were free, EC2 would win based upon features. So, the question is, does cost matter?
For us, a single engineer's salary costs more than our entire yearly EC2 bill. And, we're not even using cost saving measures like elasticity and reserved instances.
And, EC2 actually drives costs down in ways hosting does not. First, as you mentioned, it dramatically reduces the need for IT staff. (Though it doesn't eliminate it, of course.)
Second, and IMHO, more importantly, the constraints EC2 imposes on you forces you to build fault tolerant, shared-nothing systems. I'm sure many folks here have worked at companies with hosted servers to find machines that are chock full of random cron jobs, services, and so on. The instant provisioning capability of EC2 combined with the constraints makes it very hard for you to have these one off "god boxes" your system is hinging on. Beyond that, as things get too messy, you have the freedom to automate and shut-down and bring up fresh nodes instantly with no increase in cost. It's hard to me to estimate the amount of man-hours we've saved and disasters we've avoided due to this capability. It's allowed us to cleanly "refactor" our entire cluster over time since we can reprovision nodes on demand as we improve the services or architecture on those nodes.