News for hackers. Stuff that matters.
Re:The Architecture Twitter Uses to Deal with 150M Active Users by colinhowe on Monday July 08, 2013 @02:31PM (#6008300)
That's probably 6k writes to a "normal" DB. The fanout is handled by Redis which I doubt is included in the 6k writes. Not if you want to fan out to 30 million followers in under 5 seconds.
Re:The Architecture Twitter Uses to Deal with 150M Active Users by papsosouid on Monday July 08, 2013 @02:37PM (#6008337)
The "fanout" or "wasteful duplication of a single message 30 million times" is only required because they are using tiny underpowered hardware to begin with. The approach that they claim can't possibly work actually does work. You just can't do it on a $2000 "server".
Re:The Architecture Twitter Uses to Deal with 150M Active Users by kingryan on Monday July 08, 2013 @07:57PM (#6010227)
The messages are not repeated in fanout, just the ids. I guess you would know that if you actually the links.
Re:The Architecture Twitter Uses to Deal with 150M Active Users by vidarh on Monday July 08, 2013 @08:31PM (#6010366)
That still means 30 million writes instead of one. That the size of each write is smaller is not going to help you all that much if each write still worst case ends up forcing you to rewrite at least one disk sector.

While I'm inclined to favour a fanout approach, a "full" fanout can easily be incredibly costly on the write side.

Re:The Architecture Twitter Uses to Deal with 150M Active Users by sulam on Monday July 08, 2013 @07:55PM (#6010217)
Let me guess, you work for Oracle?