News for hackers. Stuff that matters.
Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by ddorian43 on Friday March 25, 2016 @01:15PM (#11361238)
Doesn't leveldb , kinda suck? Meaning every ~serious db doesn't use it or has created a fork of it ? or see the symas-lmdb benchmarks for many different usecases of different kv stores http://symas.com/mdb/#bench
Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by ivoras on Friday March 25, 2016 @02:46PM (#11361888)
It sucks allright.

The performance numbers are actually mediocre at best in "real world" conditions. When I tried using it in a project where both high performance and concurrency were needed, I also discovered that it doesn't scale past 1 CPU core - probably uses mutexes or inefficient locks. I gave up on it rather than trying to fix it.

That said, it has a simple API and sometimes neither concurrency nor ACID are important.

Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by duaneb on Friday March 25, 2016 @03:31PM (#11362129)
I believe the core writer runs on a single thread, so this makes sense.

Also, I'd argue leveldb shines in a distributed environment where the simple interface is easy to leverage at scale--after all, it is basically the tablet part of big table.

Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by cat-dev-null on Friday March 25, 2016 @05:03PM (#11362678)
Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by haberman on Friday March 25, 2016 @05:47PM (#11362915)
That appears to be a bug in munmap() on OS X that just happened to bite LevelDB.
Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by maxpert on Friday March 25, 2016 @01:48PM (#11361506)
Well LMDB is hands down the fastest of them all. All the benchmarks that I saw with LMDB were mostly on Linux and ext file system. It would be really interesting to see how the memory mapped files perform on a phone though. But doing so requires me to port complete LMDB on phone :P
Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by catwell on Friday March 25, 2016 @02:07PM (#11361639)
LMDB already runs on iOS and Android. In my experience you will have issues on some Android devices though. For instance, I recently found out that on some Android devices writes to a mmap-ed file through the usual POSIX interfaces are not immediatly reflected on reads through the mmap, which means that on those devices you MUST use MDB_WRITEMAP.

For those who don't know, I/O on Android is a nightmare. On your typical phone, the filesystem you have access to is a FUSE that mirrors to an EXT4 filesystem mounted with crazy device-specific options which include noauto_da_alloc.

I don't know any DB which works fast and consistently across all Android phones on the market. SQLite is probably the safest bet for now.

Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by mtanski on Friday March 25, 2016 @02:17PM (#11361716)
> LMDB already runs on iOS and Android. In my experience you will have issues on some Android devices though. For instance, I recently found out that on some Android devices writes to a mmap-ed file through the usual POSIX interfaces are not immediatly reflected on reads through the mmap, which means that on those devices you MUST use MDB_WRITEMAP.

How is that possible if they are using Linux and UBC (Unified Buffer Cache). It's unlikely they anybody would go undo that from the VM/FS layer.

The only think I can think off this behavior happening in a Linux app is if you're creating a private mapping... Private mappings are COW and a write to the global would cause a copy of the old data for you app. They'll be no sycing till you msync()

Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by catwell on Friday March 25, 2016 @02:55PM (#11361937)
To be honest I am not entirely sure of my interpretation of the issue yet. What I know is that on those devices you sometimes can't read your writes with LMDB if it's not using MDB_WRITEMAP.

While trying to debug I ended up on the only question ever asked by Howard Chu (LMDB author) on Stack Overflow: http://stackoverflow.com/questions/7061910/when-does-an-o-sy...

I am still trying to figure out what exactly is happening here but it's not very easy given that I do not even have access to a rooted phone that exhibits the issue.

Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by hyc_symas on Friday March 25, 2016 @07:41PM (#11363583)
It really depends on where you're storing your DB files. If you're using the /data partition it should Just Work. If you're using external storage, yes, android does a bunch of stupid tricks to implement its secure storage, and that interferes with mmap.

Here's the code that resulted from that Stack Overflow question: https://github.com/LMDB/lmdb/blob/mdb.master/libraries/liblm...

After 36 years of writing code, I have more answers than questions.

Re:Show HN: LevelDB outperforms others on a cheap phone (Mic... by hyc_symas on Friday March 25, 2016 @07:34PM (#11363544)
This whole post is kinda laughable, so a mediocre bug-ridden key/value store is somewhat faster than an SQL database. Big deal, they're all slow.

LMDB has been tested on quite a lot of filesystems, not just ext.

http://symas.com/mdb/microbench/july/#sec11

And actually, LMDB was working fine back in the Froyo/Gingerbread days. I integrated it into XDAndroid way back then. http://forum.xda-developers.com/showthread.php?t=1171052 (By way of SQLightning.)

Never tested it on things like squashfs or whatnot though.