Re:Twitter releases Gizzard, a framework for creating distri...
by nkallen on Wednesday April 07, 2010 @01:38AM
(#1246570)
I think you've given a good suggestion for upcoming blog posts.
Gizzard is used by two systems at Twitter. One of them is called FlockDB and we are working on open-sourcing it (indeed, FlockDB is why we open-sourced Gizzard). FlockDB stores Twitter's social graphs. I cannot tell you (yet) how many QPS we do do or how many edges there are in the various graphs, but suffice to say its a lot and we run lots of complicated queries like "who follows both @aplusk and @oprah but does not want any of @aplusk's retweets," etc.
Gizzard is NOT perfect. But we think you'll find it is resilient to a large class of failure scenarios (that have, in fact, occurred over the last year, so this is not just the theoretical fault tolerance of a project that's been deployed on a small web site with simple requirements). With your help, it could be even better.
Gizzard is used by two systems at Twitter. One of them is called FlockDB and we are working on open-sourcing it (indeed, FlockDB is why we open-sourced Gizzard). FlockDB stores Twitter's social graphs. I cannot tell you (yet) how many QPS we do do or how many edges there are in the various graphs, but suffice to say its a lot and we run lots of complicated queries like "who follows both @aplusk and @oprah but does not want any of @aplusk's retweets," etc.
Gizzard is NOT perfect. But we think you'll find it is resilient to a large class of failure scenarios (that have, in fact, occurred over the last year, so this is not just the theoretical fault tolerance of a project that's been deployed on a small web site with simple requirements). With your help, it could be even better.