News for hackers. Stuff that matters.
Re:How to use Amazon’s S3 web service for Scaling Image Hosting by d2viant on Wednesday July 14, 2010 @06:42PM (#1516210)
"We could push our images to Amazon, and never have to worry about backing them up, or keeping extra copies in case of hardware failure"

Perhaps a clarification, but Amazon does not guarantee your data as part of S3. They will do their best job (and have a great track record), but just like any other service -- failures will occur. Ultimately their SLA may provide for reimbursement of downtime, but that won't get your data back.

That being said, S3 is still (probably) magnitudes better than what you can homegrow.

Re:How to use Amazon’s S3 web service for Scaling Image Hosting by johnswamps on Wednesday July 14, 2010 @07:21PM (#1516300)
Related: http://news.ycombinator.com/item?id=1334187. Some guy was storing his data on Amazon EBS and it was all lost (not sure how EBS compares to S3 in terms of probability of data loss).
Re:How to use Amazon’s S3 web service for Scaling Image Hosting by timf on Wednesday July 14, 2010 @10:05PM (#1516682)
Werner Vogels says S3 is designed "for 99.999999999% durability." That is eleven nines.

(From http://www.allthingsdistributed.com/2010/05/amazon_s3_reduce... )

EBS volumes have a theoretical "annual failure rate (AFR) of between 0.1% – 0.5%, where failure refers to a complete loss of the volume."

(From http://aws.amazon.com/ebs/ )

Considering how many EBS volumes there are, that is not that low of a percentage for us to see many cases. It happens and the case at that link is not the only one.

But that person didn't take the advice, he did not snapshot his EBS volumes to S3. It says it right there in the description (as well as in the user's guide): "The durability of your volume depends both on the size of your volume and the percentage of the data that has changed since your last snapshot."

Re:How to use Amazon’s S3 web service for Scaling Image Hosting by timf on Wednesday July 14, 2010 @09:59PM (#1516664)
S3 is like a giant RAID device to me, similar caveats apply from the "RAID is not a proper backup" arguments. There is not protection against deletions (malicious or accidental), writing over a file with a corrupted file, etc.

Further, there is not protection against your account becoming errant in Amazon's eyes resulting in you getting locked out. And d2viant's point remains, Amazon could mess up or be subject to something out of their control.

I'm not trying to diminish the excellent service and good track record they have, I use it myself as a piece of my total backup solution. But just a piece.

Re:How to use Amazon’s S3 web service for Scaling Image Hosting by mattmanser on Wednesday July 14, 2010 @07:01PM (#1516256)
Hehe, I was about to post exactly the same comment.
Re:How to use Amazon’s S3 web service for Scaling Image Hosting by swindsor on Wednesday July 14, 2010 @07:01PM (#1516255)
True, there is guarantee. But it's a pretty good track record - and the underline technology holds up pretty well. It's more reliable than what we'd be able to build with a small startup budget (or pay for offsite backups/storage).
Re:How to use Amazon’s S3 web service for Scaling Image Hosting by mattmanser on Wednesday July 14, 2010 @07:04PM (#1516261)
It's a good track record unless it happens to you, and then you're screwed if you didn't do something else to save it. I'm sure at least one person on HN has posted about losing all their data 'cause a machine failed.

It's the equivalent of not getting insurance. Yes, it saves you money, but on the other hand if you're really unlucky the consequences are truly dire.

Re:How to use Amazon’s S3 web service for Scaling Image Hosting by swindsor on Wednesday July 14, 2010 @07:13PM (#1516280)
http://aws.amazon.com/s3/#protecting I am totally okay with 99.999999999% durability. This is way better than I'll be able to build/afford on my own.

In case of an epic S3 disaster, we could backup our S3 buckets to another offsite storage, but we're not going to improve much further than what their giving us.