A Detailed Five Step Twitter Scaling Plan

Re:A Detailed Five Step Twitter Scaling Plan by davidu on Wednesday May 21, 2008 @01:31PM (#196285)

I have. I have managed systems that handle millions of clicks a day in databases and billions of dynamic (PHP) pageviews on a daily basis.

I am currently managing a system that serves over 100,000 DNS queries per second, each of which is dynamically evaluated as to how we respond to it. And each of which is logged, amounting to over 7,000,000,000 log lines a day, all which get processed, parsed and in many cases handed over to a database store. Yep, we're dealing with BILLIONS of rows of MySQL per day.

[ Parent | on HN ]

Re:A Detailed Five Step Twitter Scaling Plan by Xichekolas on Wednesday May 21, 2008 @04:17PM (#196508)

Do you have anything online about your architecture? I think it'd be interesting to see how those 100k queries per second get served.

Hey thanks for OpenDNS by the way, I'm a long time user, and I love it.

[ Parent | on HN ]

Re:A Detailed Five Step Twitter Scaling Plan by davidu on Thursday May 22, 2008 @02:11AM (#196950)

I'll be speaking about it for the first time at the Velocity Conference in June. I'll have slides available.

[ Parent | on HN ]

Re:A Detailed Five Step Twitter Scaling Plan by menloparkbum on Wednesday May 21, 2008 @04:29PM (#196523)

I'm now curious as to why you switched from zeitgeist to molotovs. The yuppie hordes at Zeitgeist are extreme but it is so hard to move back indoors...

[ Parent | on HN ]

Re:A Detailed Five Step Twitter Scaling Plan by hank777 on Wednesday May 21, 2008 @01:46PM (#196311)

Nevertheless you believe that there is no way to handle a distributed in-memory lookup system. It just goes to show you cant know everything.

[ Parent | on HN ]

Re:A Detailed Five Step Twitter Scaling Plan by davidu on Thursday May 22, 2008 @02:12AM (#196951)

What do you mean? I've used distributed hash tables (in memory or otherwise) for years...

[ Parent | on HN ]