Re:Google loses data as lightning strikes
by axiak on Wednesday August 19, 2015 @12:39PM
(#10086709)
Nowadays they use Reed–Solomon coding to effectively distribute their data without copying it to 3 places.
[ Parent
| on HN ]
Re:Google loses data as lightning strikes
by Artemis2 on Wednesday August 19, 2015 @12:49PM
(#10086775)
Do you have a source for that? Why would they start using it now?
Re:Google loses data as lightning strikes
by azurezyq on Wednesday August 19, 2015 @02:11PM
(#10087351)
Disclaimer: googler, but not working on storage.
link here: http://static.googleusercontent.com/external_content/untrust...
actually in colossus one can tune RS coding parameters per file, to get a tradeoff between performance/durablity.
RS coding uses less copies, but same level of safety (tradeoff is the recovery computation time.)
Re:Google loses data as lightning strikes
by magicalist on Wednesday August 19, 2015 @01:25PM
(#10087033)
Also how would that help with geographic redundancy? Local recovery from errors does you no good if your datacenter gets wiped out in a flood.
Re:Google loses data as lightning strikes
by axiak on Wednesday August 19, 2015 @04:35PM
(#10088264)
I can't find the presentation anywhere, but at HBaseCon 2014 one of the lead developers of Bigtable stated that they went to RS. Even for their older databases.
EDIT: In this video https://vimeo.com/100153741, around the 23 minute mark