News for hackers. Stuff that matters.
Re:The Architecture Twitter Uses to Deal with 150M Active Users by thezilch on Monday July 08, 2013 @04:10PM (#6008965)
They may average 6K w/s, but history has shown higher peaks, and those peaks often include tweets from those with large 30e6+ followers -- large events naturally have celebrities chiming in. Having more CPU [cores], RAM, and IO is not going to solve for the need to shard your Redis, which is bound by a single core. For the fanout, that is anywhere from 90 to 90e3 cores at 2e6 Redis IOPs -- 1-6e3 Lady Gagas per second. Before pipelining, which you'd probably consider complex or fragile. All you've done is moved some networking parts around. Your code and topology hasn't really changed.
Re:The Architecture Twitter Uses to Deal with 150M Active Users by papsosouid on Monday July 08, 2013 @04:42PM (#6009154)
>Having more CPU [cores], RAM, and IO is not going to solve for the need to shard your Redis, which is bound by a single core

But they chose redis in the first place as part of their messy, crummy pile of barely working junkware solution. You wouldn't use redis in the first place if you were building a solution that scales up.

Re:The Architecture Twitter Uses to Deal with 150M Active Users by thezilch on Monday July 08, 2013 @07:31PM (#6010104)
So, now we also have to assume per-core software and support licenses to accompany our purchase of several $10M racks? What platform are we talking about now? Presumably I have to also purchase the IDE and toolchain licenses, per developer, too?

Twitter may have and will still make missteps, but you're being egregiously vague and unhelpful. Your comments are as vaporous as you make Twitter's architecture to be.

Re:The Architecture Twitter Uses to Deal with 150M Active Users by rozap on Monday July 08, 2013 @06:56PM (#6009929)
You're funny. I like you.