News for hackers. Stuff that matters.
Re:HFS+ Bit Rot by acdha on Tuesday June 10, 2014 @11:17PM (#7876385)
The problem is that there are other sources of error - e.g. data corruption in transit rather than on the disk itself – and the legacy methods have error rates which are too high for modern data volumes. There are a couple of implementation problems as well: the lower-level error correction mechanisms tend hide information from the higher-level interfaces, making it hard to measure real error-rates, and some classes of errors aren't randomly distributed and were more likely to produce errors which simple schemes couldn't detect.

http://queue.acm.org/detail.cfm?id=1317403 is a good article by someone at NetApp describing everything which can go wrong with hard drives, including this class of error.

There are two good papers on measured real-world error rates:

http://indico.cern.ch/event/13797/session/0/material/paper/1... http://www.cs.toronto.edu/~bianca/papers/fast08.pdf

The good news is that many of these errors were caught but there are examples which were not and the real message is that the entire stack has enough complexity lurking in it that you wouldn't want to simply assume it handles something as critical as data integrity. Something like the ZFS / brtfs approach is nice because it doesn't depend on all of those layers working as expected, is guaranteed to be monitorable and is much less likely to silently change without notice.

Re:HFS+ Bit Rot by thrownaway2424 on Wednesday June 11, 2014 @12:50AM (#7876558)
I thought one of those links was going to be "Parity Lost and Parity Regained", but no? One of them is by the same authors, on the same topic, from the proceedings of the same conference, but it's a different paper? Weird.

http://research.cs.wisc.edu/adsl/Publications/parity-fast08....

Re:HFS+ Bit Rot by acdha on Wednesday June 11, 2014 @08:46AM (#7877690)
Complete oversight on my part – thanks for the link.
Re:HFS+ Bit Rot by jsz0 on Wednesday June 11, 2014 @12:34AM (#7876520)
ZFS / births are great for their intended purposes but using them on devices with limited RAM and/or connected via external bus/power would likely introduce a whole new set of problems.
Re:HFS+ Bit Rot by yungchin on Wednesday June 11, 2014 @06:17AM (#7877190)
What's the problem with having those on external buses?