News for hackers. Stuff that matters.
Re:FlockDB: Twitter's distributed, fault-tolerant graph data... by amix on Monday April 12, 2010 @09:27AM (#1259061)
By the looks on the code this isn't really a graph database. It's more a graph database emulation built on top of a SQL database. For distribution it uses sharding so it isn't really distributed either, at least not like Cassandra or other distributed databases.

I think Redis would perform a lot better than SQL for graph like structures - since sets are a native datatype in Redis. And you can go A LONG way with just one Redis database (currently we are storing over 20 million keys in our Redis database and I know some that are storing 100 million keys on _one_ server). And with the new Redis VM coming up, I would guess that scalability of Redis is going to be even better.

Other than this, neo4j seems very interesting and would probably also have been a better choice than using a relational database.

Re:FlockDB: Twitter's distributed, fault-tolerant graph data... by lsb on Monday April 12, 2010 @10:28AM (#1259180)
You can go a long way with a SQL database too. I'm writing up an article about getting the two billion words of Wikipedia into an inverted index in a SQLite database entirely in memory, and that's another other order of magnitude bigger.
Re:FlockDB: Twitter's distributed, fault-tolerant graph data... by amix on Monday April 12, 2010 @09:07PM (#1260901)
If you only know how to use a hammer then everything else looks like a nail. In other words, re-implementing an inverted index in SQL is waste of time when you can use tools like Sphinx and Lucene - which are highly optimized to do inverted indexes and that can easily handle 2 billion words. The same can be said about FlockDB - it's possible to emulate a graph database, but is the effort really worth it when there are such tools like Redis and neo4j which seem to be optimized for graph like structures.