News for hackers. Stuff that matters.
Re:The Bw-Tree: A B-tree for New Hardware by easytiger on Wednesday April 10, 2013 @08:55AM (#5524772)
The main issue with those stats is that they are comparing sqlite3 opened liked this:

    status = sqlite3_open(file_name, &db_);
Writing to a file, with full ACID completeness and everything been put on this disk to some in memory key value stores. Completely different thing.
Re:The Bw-Tree: A B-tree for New Hardware by snaky on Wednesday April 10, 2013 @09:41AM (#5525049)
>SQLite: We tuned SQLite's performance, by setting its locking mode to exclusive. We also enabled SQLite's write-ahead logging

>Databases are opened in asynchronous write mode. (LevelDB's sync option, TreeDB's OAUTOSYNC option, SQLite3's synchronous options are all turned off, MDB uses the MDB_NOSYNC option, and BerkeleyDB uses the DB_TXN_WRITE_NOSYNC option). I.e., every write is pushed to the operating system, but the benchmark does not wait for the write to reach the disk.

By the way there is SQLite with LMDB backend - https://gitorious.org/mdb/sqlightning

>Using tool/speedtest.tcl in the SQLite source tree, the time to insert 1000 records on my laptop SSD was 22.42 seconds using the original code, and only 1.06 seconds using MDB. Both tests were run 3 times, with results average

Re:The Bw-Tree: A B-tree for New Hardware by easytiger on Wednesday April 10, 2013 @09:42AM (#5525055)
Your point eludes me? For a start that is not tuning it. It is still using synchronous IO
Re:The Bw-Tree: A B-tree for New Hardware by hyc_symas on Wednesday April 10, 2013 @11:30AM (#5525726)
Note that the LevelDB, Kyoto Cabinet, and SQLite3 configs are identical to what the Google LevelDB authors used in their original benchmark. http://leveldb.googlecode.com/svn/trunk/doc/benchmark.html

The only thing different about the Symas LMDB tests is that LMDB and BDB were added.

Re:The Bw-Tree: A B-tree for New Hardware by snaky on Wednesday April 10, 2013 @09:53AM (#5525119)
Did you look at http://symas.com/mdb/microbench/db_bench_sqlite3.cc?

Search for 'write_sync'

Re:The Bw-Tree: A B-tree for New Hardware by easytiger on Wednesday April 10, 2013 @12:48PM (#5526327)
Ahh yea i missed, apologies

     std::string sync_stmt = (write_sync) ? "PRAGMA synchronous = FULL" :
                                           "PRAGMA  synchronous = OFF";

Still. how is it a valid compare if it is getting it to ssd/hdd and the others are writing to memory? Just open sqlite3 with :memory:
Re:The Bw-Tree: A B-tree for New Hardware by hyc_symas on Wednesday April 10, 2013 @03:57PM (#5527684)
You're still not understanding. LMDB is a disk-based database. It is not only writing to memory.
Re:The Bw-Tree: A B-tree for New Hardware by easytiger on Thursday April 11, 2013 @04:49AM (#5530728)
MDB in those stats is writing to disk? I assume its some kind of lazy write though where the memory store is eventually persisted? in that case thats fairly impressive for a lot of use cases. Though a k/v store has a lot less to do than something like sqlite3.

I'll look into it (never heard of it before). Might fix fairly well with something I'm working on right now (currently implemented using sqlite3).

Re:The Bw-Tree: A B-tree for New Hardware by hyc_symas on Friday April 12, 2013 @04:30AM (#5537269)
In fully synchronous mode, which is LMDB's default mode, everything is persisted upon commit. This is a memory-mapped database, not a main-memory/in-memory database. It is fully persistent to disk, it just so happens that it uses mmap() instead of read()/write() and a user-level page cache.

People are always confusing "memory-mapped" with "memory-only". LMDB is not a memory-only database.

http://www.openldap.org/lists/openldap-devel/201111/msg00064...

Re:The Bw-Tree: A B-tree for New Hardware by easytiger on Friday April 12, 2013 @07:20AM (#5537813)
I guess (as i did) assumed it meant memory-mapped I/O as opposed to something like memory-mapped file I/O or port mapping.