Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by searchfaster on Tuesday May 09, 2017 @01:30AM
(#14297932)
> I can find the code and repost it.
Here it is.. https://github.com/yahoo/mdbm
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by luckydude on Saturday May 13, 2017 @09:08PM
(#14333745)
So Yahoo took the code and did all sorts of stuff to it.
I've got the code pre-yahoo and that's the stuff that is
fast.
It's part of BitKeeper which is open source.
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by dryman on Tuesday May 09, 2017 @12:07AM
(#14297668)
I found some performance numbers from Yahoo.
https://yahooeng.tumblr.com/post/104861108931/mdbm-high-spee...
It has random read time 0.45 μs. On my late 2013 iMac, i5 cpu, I got 0.089 μs random read. I yet to have a benchmark that measures sequential read, but the API supports it.
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by dryman on Monday May 08, 2017 @11:55PM
(#14297628)
Thanks for the reference. I'm collecting a list of embedded key-value store to benchmark against. I'll tryout with this one first!
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by corysama on Tuesday May 09, 2017 @01:15AM
(#14297885)
Fun note: Because of this line in the article
> Because MDBM gives you raw pointers into the DB’s data, you have to be very careful about making sure you don’t have array over-runs, invalid pointer access, or the like.
Someone wrote a Rust binding http://erickt.github.io/blog/2014/12/13/rust-and-mdbm/
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by dryman on Tuesday May 09, 2017 @01:41AM
(#14297959)
Wow that's a very interesting post! Maybe I should try to bring a rust binding to my project as well..
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by dom0 on Tuesday May 09, 2017 @09:20AM
(#14299735)
LMDB
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by hyc_symas on Sunday May 14, 2017 @09:40PM
(#14338545)
I would expect this hash table to be much faster than LMDB for tables that fit in RAM. It doesn't have transactions or any kind of concurrent access guarantees so naturally it has lower access overhead. For data sets larger than RAM, or in multithreaded environments, it will be pretty unusable.
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by tawayway on Tuesday May 09, 2017 @09:26AM
(#14299770)
Out of curiosity, what kind of logic analyzer are you using that can see events that fast?
Re:Writing a Very Fast Hash Table with Tiny Memory Footprints
by luckydude on Tuesday May 09, 2017 @10:24AM
(#14300233)
I dunno, it was a long time ago at SGI, maybe 23 years ago?
The CPUs were 200mhz MIPS chips. One of the hardware guys
heard me talking about trying to optimize this code and he
set up the analyzer and taught me how to use it.
I can find the code and repost it.