News for hackers. Stuff that matters.
Re:ZFS Deduplication by rbanffy on Monday November 02, 2009 @12:05PM (#917206)
Let me guess: SPARC CPUs are very fast in creating SHA256 hashes, right?

And, BTW, I am in no means implying this is bad. Quite the contrary - I would love to have an inexpensive SPARC-based desktop. If it existed.

Re:ZFS Deduplication by wmf on Monday November 02, 2009 @03:07PM (#917760)
I suspect they chose SHA256 because it's the best hash, not because it advantages their hardware. But yeah, Niagara 2 is pretty smokin for crypto: http://blogs.sun.com/sprack/entry/ultrasparc_t2_crypto_perfo...
Re:ZFS Deduplication by Andys on Monday November 02, 2009 @07:08PM (#918260)
New Intel CPUs now have a CRC32 instruction, which I am hoping Sun engineers will take advantage of soon and add as a choice for ZFS.
Re:ZFS Deduplication by Hoff on Monday November 02, 2009 @07:37PM (#918299)
VAX had a CRC instruction.

While it might seem flippant to compare VAX to x86, it was found (on VAX) that a programmer could potentially get better aggregate performance by avoiding that instruction; that having an instruction doesn't automatically mean that the code is faster.

Re:ZFS Deduplication by rbanffy on Monday November 02, 2009 @07:28PM (#918285)
CRC32? Compared to SHA256? Seriously?
Re:ZFS Deduplication by Andys on Monday November 02, 2009 @09:33PM (#918440)
I can't bring myself to trust dedupe without verification after every hash match, so I plan to use it with a fast hash algorithm with full verify enabled.

CRC32 is faster than ZFS's default of Fletcher2 and has less frequent collisions.

Re:ZFS Deduplication by Andys on Monday November 02, 2009 @09:41PM (#918455)
Many ZFS users already leave compression enabled which uses more CPU time than SHA256 and is still fast enough.
Re:ZFS Deduplication by rbanffy on Monday November 02, 2009 @02:09PM (#917596)
Why a -1?! Are x86's faster at that?