Re:How not to benchmark Cassandra: a case study
by jbellis on Thursday June 12, 2014 @04:22PM
(#7885721)
This was the toughest part to make clear as well as brief. I was not entirely successful.
For Cassandra I used the default 50ms batch window. I'm not aware of tunable windows for the others.
Re:How not to benchmark Cassandra: a case study
by _benedict on Thursday June 12, 2014 @10:49PM
(#7887223)
The difference is both Cassandra and Mongo guarantee that the data on disk is never behind by more than the sync interval (i.e. if the disk gets behind, writes block). Couchbase does not do this; it syncs periodically, but if the disk gets behind writes continue at the same rate, so the only limit to how much data can be lost is the size of your RAM.
Writing to the filesystem (without syncing) is a few hairs better than buffering in application memory, but then you are at the mercy of the filesystem blocking your party for a few hundred millis while it naval gazes at random. Not a huge deal for most workloads because you can still get great latency at 99 and 99.9, but beyond that it's a problem.
Not standardizing on a specific set of commit log behavior (sync vs async, fsync frequency) is pretty egregious for benchmarking anything with a log.
It's not clear from the blog post which direction you went in, relaxing durability in Cassandra or increasing durability in the others. I had to go to github to figure out that you switched to synchronous commits all around. That begs the question, were there other tunables necessary to make synchronous durability perform well? I know in some databases you can set how long the database will wait to group commit in terms of # of transactions and time.
Having a working group commit implementation is of course very important.