Rescuing Resque: Let's do this

Re:Rescuing Resque: Let's do this by robotmay on Sunday September 23, 2012 @03:38AM (#4560328)

I switched from Resque to Mike Perham's Sidekiq earlier in the year, and the improvement is just sensational. I'm curious to see how Resque can be improved to bring it up to the same level.

[ Parent | on HN ]

Re:Rescuing Resque: Let's do this by lucaspiller on Sunday September 23, 2012 @05:16AM (#4560460)

++ on Sidekiq. We've been using it since April and have had no issues at all. From the readme:

"Sidekiq is compatible with Resque. It uses the exact same message format as Resque so it can integrate into an existing Resque processing farm. You can have Sidekiq and Resque run side-by-side at the same time and use the Resque client to enqueue messages in Redis to be processed by Sidekiq.

At the same time, Sidekiq uses multithreading so it is much more memory efficient than Resque (which forks a new process for every job). You'll find that you might need 50 200MB resque processes to peg your CPU whereas one 300MB Sidekiq process will peg the same CPU and perform the same amount of work. Please see my blog post on Resque's memory efficiency and how I was able to shrink a Carbon Five client's resque processing farm from 9 machines to 1 machine."

[ Parent | on HN ]

Re:Rescuing Resque: Let's do this by Axsuul on Sunday September 23, 2012 @08:44AM (#4560770)

This, especially if you're doing any crawling or network-bottlenecked task. Those extra workers that you can fit within your memory footprint is money.

[ Parent | on HN ]