News for hackers. Stuff that matters.
Re:What’s New in Apple Filesystems [pdf] by throw0101a on Thursday June 13, 2019 @09:15AM (#20174499)
> Of course, but most of this kind of corruption is trivial to recover from with just retries, that's why checksumming there works, but not on the filesystem level, where it can only detect corruption.

FS-level checksum saves you from the situation when the underlying storage thinks everything is okay but it is not:

> This means there was a data error on the drive. But it’s worse than a typical data error — this is an error that was not detected by the hardware.

* http://changelog.complete.org/archives/9769-silent-data-corr...

* https://research.cs.wisc.edu/wind/Publications/zfs-corruptio...

* https://news.ycombinator.com/item?id=13851349 (via)

It's not an either-or situation, but rather both-and hardware- and FS-level checks.

Re:What’s New in Apple Filesystems [pdf] by zzzcpan on Thursday June 13, 2019 @09:29AM (#20174625)
> FS-level checksum saves you from the situation when the underlying storage thinks everything is okay but it is not:

It doesn't save you from anything, it just turns bit flips into full blocks of data loss and introduces overhead. If your application can deal with data loss it can deal with bit flips too. Filesystem-level checksumming is still useless.

Re:What’s New in Apple Filesystems [pdf] by throw0101a on Thursday June 13, 2019 @09:48AM (#20174785)
It tells you that your data is corrupted, which gives you options on dealing with the situation.

> If your application can deal with data loss it can deal with bit flips too.

No, it may not. The file may mostly be okay except for a place where a single bit flip causes the value to go from positive to negative:

* https://en.wikipedia.org/wiki/Signed_number_representations

Hardware not-handling bit flips is a documented phenomenon per my citations above. It does cause issues further up the stack. FS-level checksumming gives people the option to at least detect these errors, and depending on the situation, also perhaps fix them.

Re:What’s New in Apple Filesystems [pdf] by zzzcpan on Thursday June 13, 2019 @10:03AM (#20174939)
People already have a much better option by doing checksumming if necessary on the application level, much closer to the source of data actually with even less possibilities to miss corruption.
Re:What’s New in Apple Filesystems [pdf] by throw0101a on Thursday June 13, 2019 @10:19AM (#20175104)
Yes, doing it at the application level would be better. But doesn't preclude it doing it at other layers as well.

It's why we have Ethernet checksums, IP checksums, TCP/UDP checksums, etc. Defense in depth.

Re:What’s New in Apple Filesystems [pdf] by monocasa on Thursday June 13, 2019 @02:48PM (#20177580)
We don't have IP checksums, and only have TCP and UDP checksums because it's optional on Ethernet, and TCP/UDP doesn't always run over Ethernet anyway.
Re:What’s New in Apple Filesystems [pdf] by throw0101a on Friday June 14, 2019 @08:12AM (#20182307)