Close. Tiered, yes. But remember who we're talking about.

First, no tape. The areal storage density of tape is lower than hard disks. Too many moving parts involved. Too hard to perform integrity checks on in a scalable, automated fashion without impacting incoming work.

Second, in order to claim the durability that they do (99.999999999%), that means every spot along the pipe needs to meet those requirements. That means the "near-line HDD array" for warm, incoming data needs to meet those requirements. Additionally, if the customer has specified that the data be encrypted, it needs to be encrypted during this staging period as well. It also needs to be able to scale to tens if not hundreds of thousands of concurrent requests per second (though, for something like Glacier, this might be overkill).

They've already built something that does all that. It's called S3. The upload operations likely proxy to S3 internally (with a bit of magic), and use that as staging space.

After that, the bottleneck is likely I/O to Glacier's underlying storage - but again, not tapes. See this post for deets: http://news.ycombinator.com/item?id=4416065

Wouldn't there also need to be a lot of logic to prevent fragmentation? You'd probably want data from one user near other data from that user, i.e. on the same tape.

I'd guess that they ignore that problem and have baked the time it takes to get data from several tapes into the 3-4 hour estimate.

If you think about it, writes are more common than reads on average, so it's more efficient to just write to whatever tape is online and deal with the fragmentation problem on the read end, as opposed to queueing writes until the 'correct' tape can be brought online just save some time reading. Also in backup situations like this, it's more important to get the backup done in a timely manner.

That's true, but they're obviously using a multi-tier solution, with about a 90 day buffer before things go to tape:

  In addition, there is a pro-rated charge of $0.033 per gigabyte for items deleted prior to 90 days.

So it may just be feasible to organise writes so they end up together. But, yes, ultimately it is probably not worth it to do so.

The multiple-hour window could give you a lot of wiggle room here though. It's unlikely to take 3 hours to restore from a single tape, so even if you have to visit 2-3 tapes then you have plenty of time.

I'm sure that there is a general tiered storage platform (as mentioned above) which keeps some of the data online as well. That would let you run a "defrag" algorithm later if you find you need it.

Given it takes 3.5-4 hours to retrieve a backup before you can download it, probably not.

Thanks -- I had no idea you could just buy a tape library off the rack (more or less).

Do you think there is redundancy baked into the tape drive system?