Facebook uses Blu-ray discs with a robotic storage system for its cold storage: http://www.datacenterdynamics.com/it-networks/robot-digs-thr...
I don't know how this claim could be verified in our lifetimes.
We have an accelerated aging experiment, and we sample the part for degradation throughout the test. We also "shelf age" another test group of polymers from the same batch by placing them on the shelf.
Every year or so (or sometimes 5 year spans), we take the shelf samples and look for degradation, and compare it to the accelerated data.
In one polymer's specific case, free radicals in the polymer chain slowly react with oxygen and that's how it breaks down (takes about 10 years if the polymer isn't stable), so our accelerated aging is to put in in a pressure vessel full of high-pressure oxygen, and apply a little heat. We can get 10 years of degradation (roughly) within 2 weeks.
This is almost certainly what the archival blu-ray format has done and is doing.
Interestingly, http://blog.digistor.com/the-unparalleled-durability-of-blu-... says that there were new design constraint in blurays making them much more durable.
I don't know what Glacier uses, but the "offline" storage is much the same how mainframes use tape - disk based files (datasets in mainframe speak) are migrated to tape after a given amount of time to decrease storage costs. These datasets needs to be recalled back to disk to be used again. This is nearly identical to how Glacier operates with S3 as explained by Amazon.
The reason tape isn't available immediately is because the tape drive, unlike a traditional harddrive, doesn't contain a reader. So multiple tapes typically sit in machines with only only a few readers that move back and forth based on requests made to read the tapes. If there are many requests, the amount of time to recall a dataset increases.
Glacier may or may not be using tape, but the behavior is nearly identical to how tape has been used in past systems.
I don't know what Amazon uses, either, but I expect Glacier data to be pushed out to some layer that doesn't need power while idle at all, heavily checksummed and also replicated across the globe. The relatively heavy prices tag for accessing it probably is there to keep contention on some relatively thin pipe low. For example, if they use blue ray, they may have only a few readers online for every 100 or 1000 customers, both to keep prices low and to differentiate Glacier more from their other products.
Funny to imagine,
- My program has been stuck copying a variable for over half an hour!
- Yeah, there was a page fault and the tape loading guy still hasn't come from lunchYou rarely ever need your fire extinguisher, but that doesn't mean the LRU replacement algorithm that keeps your house clean can move them to the back of your attic.
Back in the day, before automated tape readers, tapes were stored in silos, like a library, and employees would find the cataloged tapes and manually retrieve and load them into tape readers. My company use to hire co-ops to do this back in the 70s or so.
The closest I've seen to a direct statement from Amazon is: " An Amazon statement sent to Ars says only that "Glacier is built from inexpensive commodity hardware components," and is "designed to be hardware-agnostic, so that savings can be captured as Amazon continues to drive down infrastructure costs."" http://arstechnica.com/information-technology/2012/08/for-on...
FWIW, Google's competitive Nearline service almost certainly uses disks:
The difference between Google’s Standard Storage and Nearline Storage slices on its public cloud comes down to a software layer, Tom Kershaw, director of product management for Google Cloud Platform, tells The Platform. We also think there might possibly be extra physical distance between the devices hosting the Nearline Storage and servers residing in the Google datacenters and network links that expose Cloud Platform services to the outside Internet. The Nearline Storage could also be running on slightly older equipment that is heavier on the disk drives but that is not beyond its technical or economic life.
http://www.theplatform.net/2015/03/16/why-the-coldest-storag...
https://www.google.com/about/datacenters/gallery/images/_300...
[1]https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...
https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...
Also, is it true that the service is not very popular? Are people here using it or similar?
As far as I can tell, Google's new nearline storage offering is superior to Glacier in nearly every way. The retrieval is much faster, and the retrieval costs are a flat rate per gigabyte that's equal to what Glacier gives you in the best case. Only drawback is the lack of a free tier.
1. Glacier is competing with tape. So for potential customers, if it's easier to acquire and maintain than your own tape library then that's a win.
2. In the words of James Hamilton the use case for Glacier is referred "jokingly to as Write Only Storage". If you're using Glacier for disaster recovery then paying a one time cost to retrieve a bunch of data pales in in comparison to the potential costs of not being able to recover your data.
3. Is nearline actually superior for the case of "oh shit I need to retrieve everything right now"?
4. AWS already has this cloud storage thingy called S3 and it has reduced redundancy storage if you're looking for something slightly cheaper.
http://perspectives.mvdirona.com/2012/08/glacier-engineering...
Google throttles your downloads to a fixed rate, so they're effectively forcing you to one point in that space: (3 days, $0.01/GB). With Glacier, you get a curve depending on how fast you want it. It's much more expensive at the same speed (3 days, $0.10/GB) but you can go to one extreme of getting all your data for free over the course of 20 months, or getting it all within 4 hours for about $1.80/GB.
So I guess it's fair to say that Glacier gives you more flexibility, but Google's offering is a much better value at the particular point that it optimizes for.
If it's not that much data you should have a local copy anyway. If it's massive amounts of data is your connection even that fast? And how did you wipe 50 servers at once?
Fire does that quite easily...
And why, in this day of rapid provisioning via cloud providers, do you expect a company to suffer the extended loss of waiting for new servers to want to bring the data back?
They're doing this in less than half a week, or assuming not all the backups are live data they're doing this in less than a day.
I submit that this is an extremely atypical company, and that in practice a company that just had a devastating fire is not going to be impacted more than marginally by the download speed limit.
There are a good few scenarios where there's total data loss. There are much fewer where you need to restore all the data in under a working day.
Has anyone here some more knowledge about that?