News for hackers. Stuff that matters.
Re:Improving Redis CRC performance by Scaevolus on Monday December 22, 2014 @06:49PM (#8785978)
I doubt this is as big a win as claimed. Sure, a benchmark managed to achieve 1.6GB/s CRC-64 throughput, but that requires 16KB (!) of lookup tables. If those don't reliably stay in at least L2 cache, the throughput could be very poor in practice.

If speed is important, xxHash is probably a better choice than CRC-64. http://fastcompression.blogspot.com/2012/04/selecting-checks...

If you want both speed and security, Siphash-2-4 is a cryptographically secure hash algorithm that runs at ~1.4cycles/byte for long inputs (faster than even crc16speed), and yet is still fast for short inputs (2.4cpb for 64 bytes). It also doesn't require 16KB of lookup tables. https://131002.net/siphash/ http://bench.cr.yp.to/results-auth.html#amd64-titan0

Re:Improving Redis CRC performance by seiji on Monday December 22, 2014 @06:54PM (#8786000)
The constraint was: the output must be CRC-64-Jones, so switching implementations wasn't an option.

So, given that constraint, any better solutions are welcome. :)

Re:Improving Redis CRC performance by Scaevolus on Monday December 22, 2014 @07:11PM (#8786063)
Why is that a hard constraint?
Re:Improving Redis CRC performance by fitshipit on Monday December 22, 2014 @07:55PM (#8786242)
in some cases the checksums are persisted, so future checksums must be computed in the same way.
Re:Improving Redis CRC performance by seiji on Monday December 22, 2014 @07:20PM (#8786106)
Backwards compatibility?