Re:XXH3 – a new speed-optimized hash algorithm
by injinj on Sunday March 17, 2019 @10:30PM
(#19418285)
The cache line latency is strongly associated with the number of hash call repetitions. At 1, 2, 3, 4 repeats, the latency is not present in the timestamp counter. From 5 -> 9 repeats, the latency builds, adding 2 cycles of latency each repeat step. 9 is the max latency added, for a total of 10 additional cycles. I can make the latency go away with a memory fence added after each repetition, but the total number of cycles added is about 50.
Graphs:
1. The original, with repeat at 32
2. The repeats, 1 -> 9
3. The repeats + memory fence, 1 -> 9
https://gist.github.com/injinj/138543ccc6a23ceb1fcdc05f46288...
Re:XXH3 – a new speed-optimized hash algorithm
by Cyan4973 on Tuesday March 19, 2019 @04:36AM
(#19429042)
I'm afraid I don't follow.
Is there any code source that could be read ?
The way to force the hash algorithm to actually wait for input is to use the previous hash to determine the start position of next hash's input. Jumping doesn't have to be large. You can get good results with just a few hundred bytes of variance.
Re:XXH3 – a new speed-optimized hash algorithm
by injinj on Wednesday March 20, 2019 @03:02AM
(#19439692)
This is notable, because this is what rurban/smhasher does in the small key test, which many are using nowadays.
In SpeedTest.cpp, there is a function called timehash():
uint64_t *aligned_result = (uint64_t *) aligned_alloc( 64, 256 / 8 );
uint64_t *aligned_buf = (uint64_t *) aligned_alloc( 64, 256 );
memcpy( aligned_buf, key, len );
begin = rdtsc();
for (int i = 0; i < repeat; i++) {
hash(aligned_buf,len,seed,aligned_result);
seed += aligned_result[0];
aligned_buf[0] += aligned_result[0];
aligned_buf[1] += aligned_result[1];
}
end = rdtsc();
Take away the aligned_buf += aligned_result, and the additional latency goes away.When I get a chance, I'll try your method of randomizing input.
I created a graph with these attributes:
0 total += hash (XXH364) 1 seed += hash (XXH364Seed) 2 seed += hash, val += hash (XXH364SH)
https://gist.github.com/injinj/138543ccc6a23ceb1fcdc05f46288...
There are weird bumps at some key sizes with City and XXH when the key is updated with the previous hash. My guess is there is some underlying cache line latency added when the key is written to.
smhasher should really have a test for good seeding. As I understand it, bad City seeding was the reason people started using SipHash.