Updated -- thanks.

Older reddit thread hit limit and crashed. New link

http://www.reddit.com/r/news/comments/1co395/live_updates_of...

What limits exist on a thread?

Certain threads generate enormous amounts of activity, and the reddit infrastructure will automatically place those threads into a temporary 'Read Only Mode'. This effectively ends the thread when it revolves around rapid updates, so redditors normally start 'Incident Thread 2', then 3, and so on.

During the Boston Marathon bombing, there were, if I remember correctly, over a dozen threads that continued the original conversation, simply because each of the previous threads kept getting hammered by people posting comments.

I guess I'm just curious on the technical side, about why they go into a temporary read-only mode. What issues forced that to be implemented?

I am mostly wondering whether it's the threaded nature of a thread on Reddit that makes this a difficult problem, or whether this is reflective of some limitation in the underlying datastore or something.

It's because a the database isn't completely agnosticly load-balanced, and a single story is stored/processed with more co-locality than a random sample of, say, 10000 randomly chosen comments.

It's easier to cache and serve a fixed piece of content than to constantly insert and re-compress it (which kills cached versions too)

100,000 users clicking reload every second. 500 posts per second. And there's a text capacity on an article text body.

buro9 : This is just a complete conjecture. I do not know Reddit's internal hardware infrastructure. I just know that real-time police radio transcription is the best news source, bar none. And it's getting hammered.

Interesting.

Are those are by-products of the threaded display? As a flat display of a thread would just result in new pages at the end, all of which would be incredibly cacheable.

Caching wont work for _extremely_ dynamic content like flash news update threads like these where >1 comments being posted _every_ second.