Re:ZFS Deduplication
by bensummers on Monday November 02, 2009 @10:42AM
(#916973)
Happily, they have a 'verify' mode which checks the data on disc instead of making the assumption that SHA256 works as promised.
[ Parent
| on HN ]
They do, but what I'm interested in is if it's something that actually occurs in real data (with SHA256, in the article he says that it's of more use with "worse but faster" hashing algorithms).
My initial remedy is to add another hash method and name the data based on the results of both; that way problematic data would need to trigger a collision in two different algorithms at the same time, which "should" be next to impossible. Currently my file fragments are named "{SHA256}.dat", in v2 I could instead name them "{SHA256}{SOME_OTHER_HASH}.dat".
Re:ZFS Deduplication
by jcsiracusa on Monday November 02, 2009 @12:35PM
(#917305)
Regarding the "should" part: http://stackoverflow.com/questions/1323013