News for hackers. Stuff that matters.
Re:Things I Wish I Had Known About Django Development Before... by jdunck on Wednesday April 17, 2013 @09:30PM (#5568438)
My recc after a lot of heavy use is: RabbitMQ for the queue, Redis for the result store.

RabbitMQ for the result store is madness.

Re:Things I Wish I Had Known About Django Development Before... by misiti3780 on Wednesday April 17, 2013 @10:31PM (#5568628)
why not just use redis for both - just curious ?
Re:Things I Wish I Had Known About Django Development Before... by jdunck on Thursday April 18, 2013 @03:21PM (#5572489)
Redis does OK as a queue, but not as well as rabbit.

For a start, rabbit handles OOM better. It has lower per-message overhead. It has a more flexible queueing model in general so that you can use it for both work queue-y stuff and other queuing needs. It's interesting how often queues seem handy once they are easy to use.

Then there's the fact that redis is good for lots of things other than queues, and you'll be tempted to use it for that, and that will crowd the queue use. If you hold a transaction on redis or do a large set intersection or run a lua script, you block everything else, including your fanned-out celery worker pool.