How do you learn stuff like this? Especially the JNI and off-heap memory parts.

If the keys are strings and you can use Rust, have you considered fst [0]? If the keys have prefixes in common, it is very compact. There's a blog post about it [1]

[0] https://github.com/BurntSushi/fst

[1] https://blog.burntsushi.net/transducers/

The main limitation of it is that it doesn't support removal, but if removals are infrequent you can workaround that with another fst with removed items (and periodically rebuild the whole thing)

Isn't this. this? https://swtch.com/~rsc/regexp/regexp1.html

Yes the state machine data structure are also used in regex engines. The author of the fst crate also created Rust's regex crate and the ripgrep CLI tool.

The regex crate has multiple implementations of regex matching algorithms, which is exposed as a library [0]. The implementations are selected at runtime based on which is faster and works right for a given regex. See also [1]

[0] https://blog.burntsushi.net/regex-internals/

[1] https://docs.rs/regex-automata/latest/regex_automata/#availa...

Author of fst and regex crates here.

No, it isn't. The only similarity between them is finite state machines. Cox's article is about regexes. The fst crate is a data structure for storing keys and optional values. Two completely different things.