News for hackers. Stuff that matters.
Re:Ice cold archive storage by remus on Thursday April 11, 2019 @11:02AM (#19635158)
Anyone care to speculate on the technology that allows them to offer the fast retrieval times and low cost per GB?
Re:Ice cold archive storage by puzzle on Thursday April 11, 2019 @11:09AM (#19635264)
"Fast" is relative here. It's fast compared to Glacier and others, but it's going to be slower than the more expensive tiers.

You might not need to speculate much about how it works, it's probably implemented as described by Google themselves in slides 22ff. here:

http://www.pdsw.org/pdsw-discs17/slides/PDSW-DISCS-Google-Ke...

(In a nutshell, pack some hot data with a lot of cold data on many large drives, then put a Flash-based cache in front of them to get long tail performance predictability back.)

Re:Ice cold archive storage by remus on Thursday April 11, 2019 @12:59PM (#19636628)
Thanks for the link! Interesting stuff.
Re:Ice cold archive storage by puzzle on Thursday April 11, 2019 @01:13PM (#19636804)
There was also a talk about the low level storage service and the performance isolation work that allows it to mix batch and latency-sensitive traffic on the same drive, but it doesn't seem to have been recorded: http://www.pdl.cmu.edu/SDI/2012/101112.html

Gory details are in the patents, 9781054, 9262093 and 8612990, which I'm not linking directly, because your lawyers might not approve. There's even a follow up, 10257111. It's so new, from two days ago, that Google Patents can't find it, while Justia can.

Re:Ice cold archive storage by whoisjuan on Thursday April 11, 2019 @11:37AM (#19635680)
You have to keep the data in that class for a certain period of time. That's the drawback. You can access the data at that price as long as you keep it there for a long time.

So I suspect this is not a fully cold storage. That's why they can retrieve the data faster. Seems more like an economics hack (Longer commitment to keep the data, allows them to buy and operate the storage hardware/software at a cost that can be amortized against those commitments)