HashMap was quite a bottleneck in Rust as well, for many reasons, not just the key allocation problem. But it was very easy to implement the same kind of custom hashtable, which almost by default ends up having a better memory layout than in Java.
https://github.com/mtopolnik/rust-1brc/blob/main/src/main.rs
I bet I could improve on it just a bit and match the Java time. Not sure about topping it, though.
I'd be curious to hear more. Other than the default hash function being quite expensive, and occasionally wanting to use the more expressive hashbrown API, I've been happy with Rust's std::collections::HashMap.
To be fair, I didn't try out the hashbrown API. Maybe that, together with FxHash, would have been enough for the official dataset with 97% of keys < 16 bytes. But, I was optimizing for the "10k" dataset as well, with a lot of longer keys, and hashing just the first 8 bytes was the winning idea.
That's just a custom hash function, though; couldn't you do that with the standard std::collections::HashMap?
You end up either having to do two lookups or always cloning the string even if you ended up not needing it.
https://doc.rust-lang.org/std/collections/hash_map/struct.Ha...
You might be getting confused because in that example the `HashMap` keys are references.
I know it's a cliche that someone brings up Rust in any programming thread, but I can't help myself. ;-) Several of the techniques here are easy enough in Rust that I just do them routinely.
* Rayon makes it easy to process line-wise in reasonable-sized chunks without allocating a string per line.
* The standard integer parsing stuff takes a slice, not an owned string.
* The standard hash map doesn't have the same expectation "that we pass in an instance of the key class that is equal to an existing instance"; you can call HashMap<String>::get with a &str. (One limitation is that std::collections::HashMap::entry does expect a K, so there's some redundancy if you need to insert afterward, but you could drop to hashbrown::HashMap::raw_entry_mut to avoid that.)