MDBM – High-speed database

Re:MDBM – High-speed database by justin66 on Wednesday December 10, 2014 @10:29PM (#8733045)

If you look at the MDBM page you'll notice that they currently benchmark against LevelDB, BerkeleyDB, and Kyoto Cabinet. What I was suggesting involves the same theme but better, newer competition.

I agree that it's an apples to oranges comparison in any case.

[ Parent | on HN ]

Re:MDBM – High-speed database by beagle3 on Thursday December 11, 2014 @12:51AM (#8733386)

The timings they give there can only make sense on a fast SSD or when the database they benchmark on is completely cached.

It's an apples-to-oranges comparison only of MDBM wins significantly against LMDB. If they are comparable in timing, or e.g. MDBM is 20% faster, then it would be an apples-to-apples comparison, MDBM having 20% speed advantage, and LMDB having every other possible advantage (memory safety, ACIDity, ordered retrieval, multiple databases, etc.)

LMDB is truly, incredibly, really marvelous. On 64-bit it comes close to being the end-all-be-all local KV-store. If your databases are not more than a few tens of megs each, the same is true for 32-bit processors as well.

[ Parent | on HN ]

Re:MDBM – High-speed database by rakoo on Thursday December 11, 2014 @08:25AM (#8734571)

> If your databases are not more than a few tens of megs each, the same is true for 32-bit processors as well.

Why is that ? Shouldn't 32-bit processors give you enough space in the range of hundreds of MiB ?

[ Parent | on HN ]

Re:MDBM – High-speed database by hyc_symas on Thursday December 11, 2014 @08:38AM (#8734612)

Sure. But you only have 2-3GB of usable address space, and if your app has a lot of other variables to work with, there's not much space left over.

I should note that in LMDB 1.0 we'll have dynamic unmapping and remapping, to allow 32-bit machines to work with larger DBs. (There's still a significant performance cost for this. It's only being done to allow folks to use the same code on 32 and 64.)

[ Parent | on HN ]

Re:MDBM – High-speed database by jetm9 on Thursday December 11, 2014 @11:06AM (#8735483)

Off-topic: will you have something like networked KV store aka redis. what do you think of Redis or Cassandra?

[ Parent | on HN ]

Re:MDBM – High-speed database by jbooth on Thursday December 11, 2014 @12:16PM (#8735882)

If you're interested in building a distributed DB on top of LMDB, I took care of part of the problem (replication/consistency) in my flotilla library at

https://github.com/jbooth/flotilla

Basically I'm just layering the raft consistency algorithm on top of LMDB. Both systems single-thread write transactions for consistency, so there's some mechanical sympathy. Doesn't mandate any specific data model or even a client-server network transport, it's basically a replicated embedded DB. Anyone could build replicated redis on top of it or a Cassandra clone if they want to get into managing shards/rings.

Sample app (still WIP) at https://github.com/jbooth/merchdb

[ Parent | on HN ]

Re:MDBM – High-speed database by hyc_symas on Thursday December 11, 2014 @11:22AM (#8735573)

For redis, there's already ardb, ledisdb, redis-NDS, and a few other redis clones that support LMDB. We also already have memcacheDB on LMDB, as well as HyperDex with LMDB.

I haven't looked closely at Cassandra since it's in java, and after I didn't find a simple backend plugin API I didn't look any further.

[ Parent | on HN ]

Re:MDBM – High-speed database by justin66 on Thursday December 11, 2014 @01:15AM (#8733451)

Yes, I am quite impressed by it as well.

[ Parent | on HN ]