Re:Ask HN: What's the best paper you've read in 2020?
by jeffbee on Tuesday December 08, 2020 @02:07PM
(#25349917)
I think for the major players you mentioned the 2016 paper was retrospective. Everyone was already doing it. Even mid-tier players like Dropbox Magic Pocket were using erasure coding by 2016, and their scheme was mostly written by ex-Google engineers influenced by Colossus.
[ Parent
| on HN ]
Re:Ask HN: What's the best paper you've read in 2020?
by luizfelberti on Tuesday December 08, 2020 @04:39PM
(#25351725)
Oh I am absolutely aware that erasure codes are an old thing, Reed-Solomon codes exist since the 1960's, but this is not simply a trivial application of erasure coding to a storage system: erasure codes alone don't provide the same durability guarantees that replication does. [0]
This is a combination of erasure coding AND replication, whose combined storage amplification is dramatically lower than previous SOTA.
I gave a longer explanation in a sibing comment to yours [1]
[0] http://fireless.cs.cornell.edu/publications/erasure_iptps.pd...
Re:Ask HN: What's the best paper you've read in 2020?
by jeffbee on Tuesday December 08, 2020 @05:18PM
(#25352190)
Thanks for the clarification. I still think these techniques were somewhat widespread already ... see for example this figure from US Patent 9292389 describing nested, aka layered coding that to my thinking is isomorphic with the "LRC" or "pyramid code" described by the Microsoft researchers.
By the way, not at all trying to say this paper isn't interesting. I keep it in my filing cabinet to show my colleagues when I need to describe this technique, since Google hasn't ever bothered to describe Colossus in a way I can reference.