News for hackers. Stuff that matters.
Re:GxHash is a fast and robust non-cryptographic hashing alg... by vlovich123 on Monday May 13, 2024 @12:51PM (#40345346)
Wanted to highlight it since it looks like it's a significant Pareto frontier jump from existing techniques.
Re:GxHash is a fast and robust non-cryptographic hashing alg... by not_a_dane on Monday May 13, 2024 @01:52PM (#40346136)
how do we know it is a robust one ?
Re:GxHash is a fast and robust non-cryptographic hashing alg... by ted_dunning on Monday May 13, 2024 @03:41PM (#40347456)
Did you read the article?
Re:GxHash is a fast and robust non-cryptographic hashing alg... by rurban on Monday May 13, 2024 @02:15PM (#40346434)
Haven't checked it in smhasher yet. just saw this, claiming to be the very best. Well, maybe a PR will come soon. The rust framework for hashes is prepared, there just haven't been any good rust hashes yet.
Re:GxHash is a fast and robust non-cryptographic hashing alg... by llimllib on Monday May 13, 2024 @03:12PM (#40347096)
i know nothing about this algorithm, but I went and checked that already; it does have a PR for smhasher: https://github.com/rurban/smhasher/pull/279

The note in there that the output of the algorithm is different for different SIMD widths seems to indicate that this isn't so much an algorithm as a family of related algorithms

Re:GxHash is a fast and robust non-cryptographic hashing alg... by vlovich123 on Monday May 13, 2024 @04:22PM (#40347953)
That's a bit weird since it says on the front page that:

> All generated hashes for a given version of GxHash are stable, meaning that for a given input the output hash will be the same across all supported platforms.

The github repo is more recent so maybe it wasn't stable at the time they tried to get it into smhasher from last year?

Re:GxHash is a fast and robust non-cryptographic hashing alg... by aidenn0 on Monday May 13, 2024 @06:31PM (#40349330)
Looks like the original failed smhasher, since the PR modified the compress function
Re:GxHash is a fast and robust non-cryptographic hashing alg... by ted_dunning on Monday May 13, 2024 @03:41PM (#40347453)
The article quotes results from smhasher runs.

It passes.

Re:GxHash is a fast and robust non-cryptographic hashing alg... by AlotOfReading on Monday May 13, 2024 @02:39PM (#40346710)
Honestly, the paper is pretty confusing. It's a weird mix of explaining things people reading a hash paper should already know (e.g. avalanche effect) and not explaining the important bits. The mixing stage looks a lot like a Davies-meyer hash, but it's hard to be sure without reading code. If so, this will be susceptible to fixed point attacks (among others). It doesn't particularly matter because this isn't presented as secure, but worth noting.

The paper also doesn't exhaustively verify numbers when domain=codomain, which is a super common scenario and a weird omission. 32 bits is small enough to check everything.

Re:GxHash is a fast and robust non-cryptographic hashing alg... by vlovich123 on Monday May 13, 2024 @04:19PM (#40347925)
The hash internally is a 128bit hash & can output 32 or 64 bit as well. Can you speak more to how fixed point attacks are relevant? The only reference I found online was in a cryptographic context which wouldn't be relevant here. Are other hashes like SipHash-1-3 (Rust stdlib hashmap) or XXH3 susceptible to it?
Re:GxHash is a fast and robust non-cryptographic hashing alg... by AlotOfReading on Monday May 13, 2024 @04:43PM (#40348177)
A fixed point attack would allow creating collisions. Whether that's an issue or not depends on your use case, but the answer is going to be yes surprisingly often in my experience.
Re:GxHash is a fast and robust non-cryptographic hashing alg... by singron on Monday May 13, 2024 @05:55PM (#40348960)
Can you realistically create fixed-point collisions of a randomly seeded hash function? E.g. to defeat the DOS-resistance of a hash table? I think the papers on fixed-point attacks of Davies-Meyer hashes require selecting the IV or knowing the IV in order to carry out the attack.
Re:GxHash is a fast and robust non-cryptographic hashing alg... by AlotOfReading on Monday May 13, 2024 @06:45PM (#40349452)
Not sure where the IV would come in, but both the plaintext and the initial key seem to be taken directly from the input as the first two blocks in any "compression group". Let's say you want intermediate key X after the first two steps, the first 3 blocks should be something like m_2 = D(X, E(m_1, m_0)) because we're missing the xor step from davies-meyer.

Let me know if I'm wrong though, strictly an amateur here.

Re:GxHash is a fast and robust non-cryptographic hashing alg... by singron on Monday May 13, 2024 @07:08PM (#40349645)
I took a closer look, and it looks like the compression doesn't depend on the seed at all. This makes it look like any collision is a collision on all seeds.

    pub(crate) unsafe fn gxhash(input: &[u8], seed: State) -> State {
        finalize(aes_encrypt(compress_all(input), seed))
    }
For a hash table, you would normally use just the low bits to select a bucket, so this still protects you from collisions on just a subset of the bits, but if you can produce a large number of collisions on the exact 128-bit output, then it's not DOS-resistant.
Re:GxHash is a fast and robust non-cryptographic hashing alg... by vlovich123 on Monday May 13, 2024 @04:54PM (#40348301)
I don’t know about this at all, but ChatGPT suggests that this attack is thwarted by a secure compression function and block function. As you point out elsewhere, it uses AES for compression but not sure if that’s enough. Do you know?
Re:GxHash is a fast and robust non-cryptographic hashing alg... by AlotOfReading on Monday May 13, 2024 @05:23PM (#40348646)
ChatGPT would be wrong in this case. It applies to (and was originally identified with) hash functions built from cryptographically secure hash functions. Note that this isn't as strong as AES. It's just built with some of the same machinery as AES, it doesn't carry over its guarantees.

Still an interesting construction I'll play around with some more when I have time.

Re:GxHash is a fast and robust non-cryptographic hashing alg... by oginiaux on Tuesday May 14, 2024 @06:22AM (#40353572)
Hi! I'm the author of this document. It indeed covers some concepts that will sound trivial for people experienced like you. However, I am not a cryptography researcher (only a developer passionate about performance and optimization) and nor is my entourage. So I wrote this also for people like them.

Still, I tried my best getting into the details, but I learned a lot while developing this hash function, and I still have plenty to learn especially regarding security. That is one of the reasons I made it open source: get feedback from more seasoned peers and improve this :).

Re:GxHash is a fast and robust non-cryptographic hashing alg... by AlotOfReading on Wednesday May 15, 2024 @12:51AM (#40363225)
I have to say, despite any nits I have about it the overall construction is pretty clever and I like a lot of your design choices. It's a cool algorithm.