News for hackers. Stuff that matters.
Re:ElephantDB: A database specialized in exporting KV data f... by Smerity on Friday February 18, 2011 @04:58PM (#2237463)
ElephantDB seems to be a minimalist DB made for the very specific task of serving MapReduce results from Hadoop - it doesn't even support writes to the DB (which is fair enough as then persisting data once the MapReduce results get updated would be quite painful).

The main thing is it gets rid of the painful step of having to set up another datastore, run MapReduce, take the result and store it in the database. Hopefully due to it being a purely read-only datastore solution it can also be highly performant.

The main downside is that currently they don't allow updating the data simply (though they mention they're planning to in the future). Currently to update the data you either need downtime or switch between two rings of data and update them one at a time.

I'm wondering if this was made for use in production somewhere (as many large data websites could appreciate this sort of DB {Cloudera's example site TrendingTopics[1]}) or if this is just a side project?

Edit: The primary author of ElephantDB seems to work at BackType so I can see how they could theoretically use this sort of tool.

[1] http://www.trendingtopics.org/ [2] http://www.backtype.com/

Re:ElephantDB: A database specialized in exporting KV data f... by omakase on Friday February 18, 2011 @05:11PM (#2237505)
Here's a really short slide deck with some details on how it fits into our workflow at BackType: http://www.slideshare.net/michaelmontano/designing-for-massi...

Will have lots more on our tech blog next week.

Re:ElephantDB: A database specialized in exporting KV data f... by nathanmarz on Friday February 18, 2011 @06:44PM (#2237777)
Massive amounts of BackType are served by ElephantDB. ElephantDB has very few moving parts, so it "just works". We don't spend any time dealing with operational or maintenance issues.

Adding background swapping support to the db is high priority for us, so that downside should go away soon.

Re:ElephantDB: A database specialized in exporting KV data f... by coderdude on Friday February 18, 2011 @05:08PM (#2237494)
It sounds like it might be for use in production. From the readme: "BackType uses ElephantDB to export views from TBs of data and serve them in the analytics applications and APIs of backtype.com and backtweets.com."