News for hackers. Stuff that matters.

A comment on ZFS Deduplication. Parent comment

Re:ZFS Deduplication by Andys on Monday November 02, 2009 @07:08PM (#918260)
New Intel CPUs now have a CRC32 instruction, which I am hoping Sun engineers will take advantage of soon and add as a choice for ZFS.
[ Parent | on HN ]
Re:ZFS Deduplication by Hoff on Monday November 02, 2009 @07:37PM (#918299)
VAX had a CRC instruction.

While it might seem flippant to compare VAX to x86, it was found (on VAX) that a programmer could potentially get better aggregate performance by avoiding that instruction; that having an instruction doesn't automatically mean that the code is faster.

[ Parent | on HN ]
Re:ZFS Deduplication by rbanffy on Monday November 02, 2009 @07:28PM (#918285)
CRC32? Compared to SHA256? Seriously?
[ Parent | on HN ]
Re:ZFS Deduplication by Andys on Monday November 02, 2009 @09:33PM (#918440)
I can't bring myself to trust dedupe without verification after every hash match, so I plan to use it with a fast hash algorithm with full verify enabled.

CRC32 is faster than ZFS's default of Fletcher2 and has less frequent collisions.

[ Parent | on HN ]