Re:As internet forums die off, finding community can be hard...
by jedberg on Thursday November 19, 2020 @05:54PM
(#25155329)
I can tell you why reddit does it. Because their entire architecture is built around new data being hot and old data not.
The tiny tin-pot hobby forum can keep every single post in memory and it's not really a problem. They can also do a full database scan pretty quickly.
But reddit can't do that. If it's not in the cache, it takes a lot of work to pull the data from the database. And there is no way to cache the entire dataset. That's why threads get locked at 6 months. So they can be statically archived for quick access.
Economies of scale isn't really part of it, it's more about moore's law.
The tiny tin-pot hobby forum can keep every single post in memory and it's not really a problem. They can also do a full database scan pretty quickly.
But reddit can't do that. If it's not in the cache, it takes a lot of work to pull the data from the database. And there is no way to cache the entire dataset. That's why threads get locked at 6 months. So they can be statically archived for quick access.
Economies of scale isn't really part of it, it's more about moore's law.