News for hackers. Stuff that matters.
Re:What New Twitter Employees Do For The First Week by ck2 on Monday February 07, 2011 @11:58PM (#2192004)
That's a good breakdown of the processes but I am positive all that is done in stages and queues.

So with a good design you have groups of servers doing the different stages in the queue.

One you've got the pattern down for 100 tweets per second, the pattern should be reproducible by scaling servers in each queue to 1000 tweets per second, and eventually 10,000 tweets per second.

The database requirements may explain why it's all done in one datacenter instead of trying to do replication across the country/world.

Re:What New Twitter Employees Do For The First Week by jacques_chester on Monday February 07, 2011 @11:59PM (#2192012)
In the research literature this is called SEDA: http://en.wikipedia.org/wiki/Staged_event-driven_architectur...