News for hackers. Stuff that matters.
Re:Fast Perfect Hashing by munificent on Wednesday February 13, 2019 @01:32PM (#19155387)
This article is interesting but... weird. It seems to assume a bunch of things, and then states some conclusions based on that, but then also assumes the value of the conclusions is obvious.

> A perfect hash function is one that is collision-free. By implication, the hash must be at least as many bytes as the key and the function is theoretically reversible, though not always tractably so.

This is true because of the pigeonhole principle, but most people won't assume that the key size is equivalent to the domain size. Usually when using perfect hashing, you are hashing (much) fewer elements than the total range a key can represent.

A perfect hash function that uniquely assigns hash values to the eight items you need to store, but gives you back integers anywhere in the 32 bit range isn't super helpful. Or, at least, it's not obvious to me why it would be. They do state the algorithm can be scaled down to 8 or 16 bits, so I guess there's that.

> Because the hash is no smaller than the key, the primary use case is randomizing small values like integral types.

This just doesn't make sense to me. The key can be much larger than the hash, even with perfect hashing, as long as the number of keys in your data set is smaller than the maximum hash value.

The author says "primary use case" but what they really mean is that this isn't what most people would consider a perfect hash at all. It's just an integer permuter. In other words, here as an even faster implementation of what the author would describe as a perfect hash over small integer keys:

    uint32_t hash(uint32_t key) { return key; }
This also guarantees there are no collisions so is a "perfect hash". It's just not a very useful one.

The author assumes the reader knows the value of randomizing your integer keys. I'm not sure what that value is in cases where you know you don't have collisions anyway.

Re:Fast Perfect Hashing by derefr on Wednesday February 13, 2019 @02:45PM (#19156197)
> The author assumes the reader knows the value of randomizing your integer keys. I'm not sure what that value is in cases where you know you don't have collisions anyway.

If you're going to be doing writes of your data in order of an integer sequence number, and you're writing to a multi-node storage system that partitions/shards by primary-key range, then you're going to get worst-case performance if you use your integer sequence numbers directly as the primary keys, since you're queuing all your writes onto one shard.

Instead, you want to permute your integer sequence numbers so that successive integers become evenly spread across the range partitions. This (along with some write duplication and verify-on-read) is the foundation of Dynamo-based "scale-free" storage systems (DynamoDB+S3, Riak+CS, etc.)

Re:Fast Perfect Hashing by jnwatson on Wednesday February 13, 2019 @04:20PM (#19157126)
It is trivial to generate a mostly-randomized bijective mapping of small integers to small integers with an xor and a little byte swapping, which is what the author does with a built-in AES instruction.

The OP's point is that this isn't a particularly interesting problem. The interesting problem is generating efficient bijective mappings of large integers to very small integers.

Re:Fast Perfect Hashing by derefr on Wednesday February 13, 2019 @06:57PM (#19158372)
I wasn't attempting to argue the OP's point, merely answering the question posed: "what [is the] value [of randomizing your integer keys] in cases where you know you don't have collisions."
Re:Fast Perfect Hashing by munificent on Thursday February 14, 2019 @12:06AM (#19159753)
Thanks, that's an interesting one I'm not familiar with.
Re:Fast Perfect Hashing by bjourne on Wednesday February 13, 2019 @02:30PM (#19155994)
The identity function is a perfect hash function, no quotes needed. Several programming languages uses it as the default function for hashing integers. But it doesn't have a good avalanche which is important for some use cases. You could, for example, use it to make guessing urls harder.

https://en.wikipedia.org/wiki/Avalanche_effect