News for hackers. Stuff that matters.
Re:Google loses data as lightning strikes by bsurmanski on Wednesday August 19, 2015 @11:53AM (#10086371)
As mentioned in the article, it only affected recently written data of 'Google Compute Engine' services. GCE allows user to launch VMs and generate arbitrary data on the server.

Normally, Google redundantly distributes out data to at least 3 different geographically distinct locations. Check out the 'BigTable' white paper [0] for more info.

For 99% of cases (and pretty well all user cases), this would not cause data lose. The key here is that the data was generated on the servers and did not have a chance to duplicate before the event.

[0] http://research.google.com/archive/bigtable-osdi06.pdf

Re:Google loses data as lightning strikes by axiak on Wednesday August 19, 2015 @12:39PM (#10086709)
Nowadays they use Reed–Solomon coding to effectively distribute their data without copying it to 3 places.
Re:Google loses data as lightning strikes by Artemis2 on Wednesday August 19, 2015 @12:49PM (#10086775)
Do you have a source for that? Why would they start using it now?
Re:Google loses data as lightning strikes by azurezyq on Wednesday August 19, 2015 @02:11PM (#10087351)
Disclaimer: googler, but not working on storage.

link here: http://static.googleusercontent.com/external_content/untrust...

actually in colossus one can tune RS coding parameters per file, to get a tradeoff between performance/durablity.

RS coding uses less copies, but same level of safety (tradeoff is the recovery computation time.)

Re:Google loses data as lightning strikes by magicalist on Wednesday August 19, 2015 @01:25PM (#10087033)
Also how would that help with geographic redundancy? Local recovery from errors does you no good if your datacenter gets wiped out in a flood.
Re:Google loses data as lightning strikes by axiak on Wednesday August 19, 2015 @04:35PM (#10088264)
I can't find the presentation anywhere, but at HBaseCon 2014 one of the lead developers of Bigtable stated that they went to RS. Even for their older databases.

EDIT: In this video https://vimeo.com/100153741, around the 23 minute mark