S3 was down unscheduled more in 2008 than we were. (I only know about their major published widely outages, and know about all of ours that lasted more than 15 seconds.)

I realize that's not a perfect indicator, maybe it was an especially bad year for S3/good year for us, nor is their app apples-to-apples against ours, but one of the primary reasons we CAN'T afford to go even to S3 for "cheap" file storage is that it represents a serial reliability problem: both "them" and "us" have to be up for us to make money.

When we price it out, the 3-year fully burdened cost of our in-house storage vs S3 pricing is about a wash. (It's 5-10% in favor of in-house, but I'm sure we're not perfectly fairly accounting for all the tiny costs [someone has to file purchase orders, do shipping/receiving, calculate depreciation and file tickets to let EMC techs into our DCs to replace failed drives, etc, etc] in our model, but that is after accounting for the IT team dedicated to storage and the on-call rotation portion allocated to storage.)

If we were a lot smaller, it would be a no-brainer to try to use cloud services where it made sense, but as Tom said and anamax clarified, once you're at scale and have access to good IT staff in-house, the cloud offerings are not particularly compelling cost-savings measures.

(Edit: I should perhaps clarify. We do use S3 now, in production, for one of our offerings that is not core to our business, and one for which an arbitrary 8 hours of downtime would not present a problem. We also built two years ago the ability to overflow our bulk/cold image uploads to S3 if we run out of space on our in-house storage. Here again, if we suddenly find ourselves having substantial unforecast demand, it's better to go out to a third party than to turn customers away because the disk is full.

99.90% is >>> 0.00%. :) )