News for hackers. Stuff that matters.

A comment on S3QL: an S3 FS with encryption, de-dup, immutable trees and snapshotting. Parent comment

Re:S3QL: an S3 FS with encryption, de-dup, immutable trees a... by cmurf on Tuesday September 01, 2015 @03:52PM (#10155232)
How do you detect bit flips with software RAID? A scrub can detect differences, but without checksums it's ambiguous which copy is wrong. And in the case of mdadm, which has no concept of a file system, it won't report the affected file. I'm not even sure you get an LBA, but if you do, then you have to go look that up, accounting for offsets, with the chosen file system to get a file.

Conversely, Btrfs scrubs will report a corrupt file path if there's a bit flip in a single copy. If there's another copy available then there's just a kernel note that there was a data csum error and the problem was fixed, no file name path.

[ Parent | on HN ]
Re:S3QL: an S3 FS with encryption, de-dup, immutable trees a... by pslam on Wednesday September 02, 2015 @12:40PM (#10160443)
The entire point of my system is that I don't need to care which copy is good. If it scrubs and detects any error, I'll mount disks independently as two filesystems, and diff the contents. I'll then restore corrupted data from backup.

Yes, a checksumming filesystem would be better, but at the time of creation (and even now?) none of the filesystem choices were mature, or proven to be sufficiently reliable. Ext4 and software RAID1 are both mature and proven reliable.

[ Parent | on HN ]