News for hackers. Stuff that matters.
Re:Twitter responds to Techcrunch about its scaling issues by chrisbolt on Sunday June 01, 2008 @03:17AM (#205727)
> We currently use one database for writes with multiple slaves for read queries. As many know, replication of MySQL is no easy task, so we've brought in MySQL experts to help us with that immediately.

This really makes me wonder if they know what they're doing. MySQL replication really isn't that hard at all. They're hitting performance bottlenecks with only 3 servers and don't appear to be looking into sharding/partitioning to split up writes?

Re:Twitter responds to Techcrunch about its scaling issues by swombat on Sunday June 01, 2008 @06:45AM (#205799)
On the contrary, I'd say that they must really know what they're doing. How many sites with a million daily active users do you know that run on only 3 db servers?

The twitter team deserves a lot of respect for managing to stretch their hardware so far.

Re:Twitter responds to Techcrunch about its scaling issues by gaius on Sunday June 01, 2008 @09:25AM (#205852)
smirk The company I work for serves volumes that dwarf that of Twitter. We don't drop anything, because we are handling actual money, not tweets. And we get by with 3 database servers, one of which is for failover.

OK admittedly they are wardrobe-sized Suns running Oracle, but that's not the point. Yes it takes skill to do a lot with a little, but it takes skill not to paint yourself into a corner too, and you need both skills to succeed.

Re:Twitter responds to Techcrunch about its scaling issues by swombat on Sunday June 01, 2008 @11:32AM (#205911)
I'd point out that 3 wardrobe-size Sun servers running Oracle probably cost about as much as 300 MySQL servers like Tweeter is running, but some might say I'm the one being a smartass then :-P
Re:Twitter responds to Techcrunch about its scaling issues by jonknee on Sunday June 01, 2008 @06:27PM (#206228)
Twitter definitely does not have millions of daily active users. There are only 824M total tweets. Comparing IDs from posts 24 hours apart shows there were ~500,000 tweets in the past 24 hours.
Re:Twitter responds to Techcrunch about its scaling issues by swombat on Monday June 02, 2008 @04:40AM (#206561)
I didn't say "millions", I said "a million". That's according to their own reports, and if there's 500'000 tweets, it's quite likely there's a million people sitting there with a tweeter client open (not everyone tweets every day).

Daniel

Re:Twitter responds to Techcrunch about its scaling issues by jonknee on Monday June 02, 2008 @10:38AM (#206738)
Your guess is still way high. It's around 200,000 active per week. Probably close to that every day as well since it's a dedicated group.

http://www.techcrunch.com/2008/04/29/end-of-speculation-the-...

Re:Twitter responds to Techcrunch about its scaling issues by briansmith on Sunday June 01, 2008 @03:37AM (#205735)
It looks to me like he is saying they are planning on ditching MySQL altogether. That is probably why they don't want to invest in redoing/expanding the MySQL replication.
Re:Twitter responds to Techcrunch about its scaling issues by chrisbolt on Sunday June 01, 2008 @03:51AM (#205738)
Twitter isn't that complicated of a website, it shouldn't take them more than a few days to migrate only the high load sections (tweets) to a sharded architecture. All they need is money for more database servers (and they have at least $15 million) and know-how. And yet they're planning to ditch MySQL in favour of... text files? And they can't even make a small MySQL cluster highly available?
Re:Twitter responds to Techcrunch about its scaling issues by tonystubblebine on Sunday June 01, 2008 @04:23AM (#205756)
Here's an interesting proposed architecture that involves ditching MySQL: http://randomfoo.net/blog/id/4182