News for hackers. Stuff that matters.
Re:Ask HN: Should I quit the field of software development? by varunsaini on Sunday September 13, 2020 @09:05AM (#24460587)
Twitter has changed this approach few times I guess, earlier it used to be simply insert tweet into a collection of tweets, and then when you load use timeline, look up the people they follow and find/merge those tweets. But it's going to create a lots of load on systems. Another approach is to maintain a cache of user's timeline(mailbox of tweets), when user posts a tweet, lookup all the people who follow that user, and insert the tweet into each or their timeline cache. results have be pre-computed, so less load. Both approaches fails when you have folks with lots of followers, so may be they use a hybrid of these approaches. this is Discussed in detail in "Designing Data-Intensive applications" book.
Re:Ask HN: Should I quit the field of software development? by jsjohnst on Sunday September 13, 2020 @03:20PM (#24463219)
> Both approaches fails when you have folks with lots of followers

The traditional mailbox architectural model can be improved via a few methods though to increase scale.

1. The entire tweet doesn’t need to be duplicated for each follower, only a lookup reference (“ID”) to the tweet.

2. Unlike in a traditional mailbox sense, each user’s timeline is a bounded collection. So rather than maintaining every tweet for every followed user in the timeline, it’s capped to the X most recent.