Re:ZFS Deduplication
by bensummers on Monday November 02, 2009 @01:23PM
(#917445)
It's not a useful measure because it's not the same thing at all. Dedup only works if the block is aligned to a ZFS block size. Compression will find blocks which have no particular alignment.
If they were same thing, you'd get the exact result, but not an approximation.
tar.x will compress chunks of data that is smaller than zfs block size. ZFS dedup + ZFS compression will compress them as well, so what's the problem?
While it might be true for some compression formats/programs, it is not the case for .tar.*, as the directory is archived to the single file first (tar), and then compressed. So if you have similarities within 2 different files, that will be exploited.
I think to make something work for "large blocks" that already does a good job for "small chunks" is just finding the right values for parameters of compression algorithm used.