News for hackers. Stuff that matters.
Re:Fleur – A bloom filter implementation in C by llimllib on Wednesday July 27, 2022 @05:16PM (#32257091)
Neat! Because it's a thing I do[1], the hash algorithms in use:

Fleur (and DCSO/bloom and DCSO/flor): fnv

bloomd: a combination of SpookyHash and murmur[2]

[1]: https://llimllib.github.io/bloomfilter-tutorial/ (I'll update it to add bloomd and spooky)

[2]: https://github.com/armon/bloomd/blob/23c19a7f5cbb35d7c3d970b...

[3]: I have a vague recollection of somebody telling me why combining two hashes in the way bloomd does for k >= 4 is a good idea but I can't remember - anybody have a good reference for me to link to? (edit: nvm, I already link the paper on my page! sheesh)

Re:Fleur – A bloom filter implementation in C by FreakLegion on Wednesday July 27, 2022 @05:30PM (#32257272)
It's in the source, Less Hashing, Same Performance: Building a Better Bloom Filter: https://www.eecs.harvard.edu/~michaelm/postscripts/rsa2008.p...

I've used this trick at scale on network gear and it works great.

Re:Fleur – A bloom filter implementation in C by ascar on Wednesday July 27, 2022 @06:13PM (#32257702)
I vaguely remember having read about multiple hashes for bloom filters in the past, but I have trouble extracting the actual approach from the linked paper.

Could you give a summary? The paper is quite mathematical and seems to lack a clear description of how to actually use the two hashes without reading in depth.

Re:Fleur – A bloom filter implementation in C by FreakLegion on Wednesday July 27, 2022 @07:00PM (#32258189)
The basic idea of a Bloom filter is that you represent each item in the filter by setting k bits, and you set and query these bits using k hash functions. The bits need to be independent and uniformly distributed, so the hash functions need to output suitably random values for the same input.

Even fast hash functions like Murmur add overhead, though, and the lower the desired false positive rate, the more hash functions you need (x hash functions for 2^-x false positive rate).

The conclusion of the paper is roughly that you can create new hash functions by recombining the outputs of two initial hash functions without compromising the statistical integrity of the filter, and this makes querying the filter a lot faster.

To be clear, even the two initial hash functions can be halves or quarters of a single hash function with an output larger than the filter needs, e.g. a filter that needs 64-bit hashes can run entirely on Murmur-128 using its bottom and top halves as the two hash functions.

Re:Fleur – A bloom filter implementation in C by llimllib on Wednesday July 27, 2022 @07:00PM (#32258195)
The bloomd source I linked above provides an excellent simple practical implementation
Re:Fleur – A bloom filter implementation in C by llimllib on Wednesday July 27, 2022 @05:38PM (#32257363)
hah yeah I noticed a minute later, the link is right in the comments, and I'd already linked the paper in my article! I feel like this is one of those things I re-learn every 5 years