News for hackers. Stuff that matters.
Re:How Facebook stores its billions of photos by ars on Thursday April 30, 2009 @09:00PM (#587855)
I wonder if they have some mechanism for rebuilding the raid array periodically to handle the inevitable undetected read errors. With raid6 you can actually do this, by comparing both parity blocks with the actual data.

Hard disks typically have undetected read error rates of 1 bit in 1E15, so assuming they transfer 1PB per day that's about 9 errors per day. Which isn't much I guess, but I wonder if they do anything about it.

Re:How Facebook stores its billions of photos by kmavm on Friday May 01, 2009 @12:26AM (#588082)
(Not speaking officially, obviously. I had nothing to do with Haystack anyway).

They're images. So you flip a bit every once in a great while; as long as the bit isn't part of image metadata, no human user will notice. The next time they load the image, (at least, assuming it's been long enough for it to get evicted from the CDN) they see the right data.