News for hackers. Stuff that matters.
Re:LevelDB for Ruby by kposehn on Sunday June 23, 2013 @02:30PM (#5928748)
Interesting.

The snap! feature is something that could be potentially useful, along with the batch feature.

We currently deal with millions of rows a minute (about 10-16k/second) on our platform and need to store, queue and process hashed data across multiple systems. Redis is our main system for it, but some of the data needs to be archived and doesn't need to remain in memory, but still needs to eventually be processed.

Could be good for that. Thanks for posting!

Re:LevelDB for Ruby by krenoten on Sunday June 23, 2013 @03:42PM (#5929082)
One thing to keep in mind with LevelDB is that it has been created with web browsers and mobile devices as the main use case, NOT as a high-performance data store. If the features of LevelDB appeal to you, however, you may want to check out Basho's fork of it that they have tuned for use in a server environment: https://github.com/basho/leveldb . If this is interesting to you, check out Matthew Von-Maszewsky's recent talk at the Ricon East conference about his work tuning it: http://www.youtube.com/watch?v=vo88IdglU_8
Re:LevelDB for Ruby by rescrv on Sunday June 23, 2013 @07:49PM (#5930090)
There's also the HyperLevelDB fork which outperforms the Basho one by quite a bit for random write workloads. Here's a write up about it: http://hackingdistributed.com/2013/06/17/hyperleveldb/
Re:LevelDB for Ruby by DAddYE on Sunday June 23, 2013 @10:39PM (#5930668)
basho and hyperlevel should be two easy targets for this gem, this week I'll plan to adapt the code (if necessary)
Re:LevelDB for Ruby by rvagg on Sunday June 23, 2013 @11:28PM (#5930828)
This is such a silly meme. It IS a high performance data store, forget what you THINK it is/was designed for or what people say about it in that respect, rely on actual data to see if it works for your data & load. There's no competition if you want to compare it to an external data store it'll beat the alternatives easily. Comparing to other embedded data stores and the 2 main forks (basho and HyperDex) and alternatives like MDB is a different matter and this is where the nature of your data, the way you access the store (single thread? multi?) and your real load are much more relevant than heresay and benchmarks that bear no relation to your actual needs.
Re:LevelDB for Ruby by kposehn on Sunday June 23, 2013 @05:47PM (#5929651)
Thanks for posting the follow-up info on tuning!