News for hackers. Stuff that matters.
Re:Riak 1.0 by geoffhill on Tuesday September 27, 2011 @04:26PM (#3045061)
LevelDB support! Would love to see how it compares to Bitcask in terms of speed.
Re:Riak 1.0 by willbmoss on Tuesday September 27, 2011 @05:55PM (#3045421)
Bitcask can guarantee one disk seek, whereas LevelDB will do one disk seek per level, so at least from that perspective, it can't be better.

Level also has to look down the entire tree if a key is missing. This means inserts end up being more expensive than reads or updates (which are all just a hash lookup in Bitcask).

Re:Riak 1.0 by fizx on Tuesday September 27, 2011 @07:28PM (#3045734)
"Bitcask can guarantee one disk seek, whereas LevelDB will do one disk seek per level, so at least from that perspective, it can't be better."

Yep, this is a standard tradeoff. When you want your data to be iterable, you have to take the hit. In practice (I oversee a large cassandra cluster), this hit happens about ~1% of the time, which is either a lot, or a little, depending on your constraints.

"Level also has to look down the entire tree if a key is missing."

This is why Cassandra has a bloom filter on top of a very similar data store.

Re:Riak 1.0 by nosequel on Tuesday September 27, 2011 @07:15PM (#3045704)
LevelDB is there as the replacement for those who are currently using Innostore as their backend and not for those who have a dataset that fits bitcask.
Re:Riak 1.0 by Ixiaus on Tuesday September 27, 2011 @05:08PM (#3045234)
I imagine it won't be as fast? The cool thing about Bitcask is that all of the keys are in memory - I imagine that would also be beneficial with secondary indexes now supported...

LevelDB seems mostly well suited for data that becomes (in terms of key size and number of keys) bigger than your RAM...

Re:Riak 1.0 by dsl on Tuesday September 27, 2011 @05:39PM (#3045344)
I think it will be a welcome change for anyone who runs a decent sized Riak deployment. We are currently adding machines simply to increase available RAM in the cluster.
Re:Riak 1.0 by nirvana on Tuesday September 27, 2011 @07:08PM (#3045672)
Why not bring a node down, and then replace it with a node that has more RAM? Are you exceeding the size of a node you can supply (in terms of RAM) for your cluster?

I'd be very curious to know a bit about the character of your data, the size of your cluster, etc. (I've only run test clusters at this point, so hearing from someone doing production work would be informative.)

Re:Riak 1.0 by dsl on Wednesday September 28, 2011 @06:19PM (#3050151)
Replacement vs. addition is a situational trade-off, but ultimately the problem remains that you need to bring more RAM to the party.

My biggest RAM consumer stores historical data for a goods trading platform. Each trade is a unique key, with all the trade data being the value. Access speed is important, but not as critical as the other goodies I get from Riak (replication and automated rebalancing). Metadata is stored separately, but I hope to change that with Riak 1.0 secondary indexes.

Re:Riak 1.0 by varworld on Tuesday September 27, 2011 @05:56PM (#3045426)
Secondary indexes are currently only supported on levelDB.
Re:Riak 1.0 by Ixiaus on Tuesday September 27, 2011 @10:01PM (#3046121)
Interesting - well, that would get me to choose leveldb then!