News for hackers. Stuff that matters.
Re:I Broke Justin.tv by justin_vanw on Tuesday July 19, 2011 @11:11PM (#2784152)
Every hotshot thinks they don't need tests until they try writing tests for a while, then they usually become evangelists.

My shock and amazement comes from the 24 hours to restart your payments system? Whaaaaat? If that is correct you guys have far, far more issues than not having tests.

Re:I Broke Justin.tv by emmett on Wednesday July 20, 2011 @12:11PM (#2786218)
How do you restart a system which holds open long-lived connections without kicking everyone off?

If you want to be kind to your users, you stop accepting connections on a new server, wait a period of time (the longer the better), and then restart. It's a matter of how many users you're willing to disrupt for how quickly you can restart the system.

The canonical example at Justin.tv is the video system: people broadcast for days to weeks at a time. If you restart the server they're connected to, their stream will be disrupted (even if the auto-reconnect works).

We have a separate system that handles most of the complexity which is stateless, but sometimes you need to restart the actual connection-holding-daemon itself. How would you suggest doing that without disrupting service?