News for hackers. Stuff that matters.
Re:Ask HN: What's a good, open source, distributed, worker q... by derwiki on Monday September 20, 2010 @02:54AM (#1708046)
A decent set up is multiple job servers and having worker daemons for the difference queues on multiple machines. As long as you have at least one job server and worker daemon going, you're in business (and you should probably put a Nagios alert on that stuff anyway). It's naive balancing, but it will taking you surprisingly far (we've been using this in production for about 6 months). That said, it's got it's own set of quirks: job servers accepting jobs when they're unable to process them while they replay a transaction log for starters -- and there's a few other subtle "job complete" bugs we haven't been able to figure out yet.

There are no comments on this comment yet.