Re:Disks lie. And the controllers that run them are partners...
by rarrrrrr on Saturday September 08, 2012 @12:20AM
(#4492223)
At SpiderOak, every storage cluster writes new customer data to three RAID6 volumes on 3 different machines, which wait for fdatasync, before we consider it written.
... and before that begins, the data is added to a giant ring buffer that records the last 30 TB of new writes, on machines with UPS. If a rack power failure happens, the ring buffer is kept until storage audits complete.
I think every company that deals with large data they can't lose develops appropriate paranoia.
The behavior of hard drives is like decaying atoms. You can't make accurate predictions about what any one of them will do. Only in aggregate can you say something like "the half life of this pile of hardware is 12 years" or "if we write this data N times we can reasonably expect to read it it again."
Re:Disks lie. And the controllers that run them are partners...
by baruch on Saturday September 08, 2012 @03:15PM
(#4493706)
Interesting, do you do disk scrubbing to make sure the data is still out there after a while? That would be the key method to ensure you can still recover the data upon need.
... and before that begins, the data is added to a giant ring buffer that records the last 30 TB of new writes, on machines with UPS. If a rack power failure happens, the ring buffer is kept until storage audits complete.
I think every company that deals with large data they can't lose develops appropriate paranoia.
The behavior of hard drives is like decaying atoms. You can't make accurate predictions about what any one of them will do. Only in aggregate can you say something like "the half life of this pile of hardware is 12 years" or "if we write this data N times we can reasonably expect to read it it again."