News for hackers. Stuff that matters.
Re:Rescuing Resque again (2015) by AznHisoka on Friday January 15, 2016 @11:57AM (#10910476)
When i started my project, I used Resque but almost immediately switched to Sidekiq a few months later. I understand it's almost impossible to run into thread safety issues with Resque, but if you're doing anything CPU intensive, Sidekiq really offers much higher performance. The maintainer also makes a living off Sidekiq, selling it to businesses and enterprises too so you can expect more frequent updates.
Re:Rescuing Resque again (2015) by otterley on Friday January 15, 2016 @12:04PM (#10910524)
> but if you're doing anything CPU intensive, Sidekiq really offers much higher performance

The threading model (LWPs vs. forking) should have no impact whatsoever on how quickly a CPU-intensive task completes.

Re:Rescuing Resque again (2015) by andrewvc on Friday January 15, 2016 @12:25PM (#10910685)
Unless it's a short-lived CPU intensive task, in which case in matters a great deal.
Re:Rescuing Resque again (2015) by otterley on Friday January 15, 2016 @06:07PM (#10912962)
The overhead of fork() is negligible, well under 0.1ms.
Re:Rescuing Resque again (2015) by andrewvc on Friday January 15, 2016 @10:31PM (#10914101)
There's also the issue, in ruby, that CoW doesn't really work due to the fact that the GC dirties all pages.
Re:Rescuing Resque again (2015) by mperham on Friday January 15, 2016 @10:46PM (#10914144)
This was fixed in Ruby 2.0.
Re:Rescuing Resque again (2015) by AznHisoka on Friday January 15, 2016 @01:40PM (#10911174)
It matters how many of these tasks can be completed in a fixed amount of time. Creating 50 processes requires more resources than 1 process with 50 threads.
Re:Rescuing Resque again (2015) by sanderjd on Friday January 15, 2016 @02:37PM (#10911552)
Yeah, Sidekiq is an obvious switch once you find yourself running 20 500MB Rails processes just because you want to be able to handle a few simple tasks concurrently. If you have a few big, long-running, CPU-intensive background jobs, then the benefits are less clear, but if you have lots of small, short running jobs where you're mostly interested in throughput, then Sidekiq makes a lot more sense.
Re:Rescuing Resque again (2015) by otterley on Friday January 15, 2016 @06:05PM (#10912953)
Resque-pool (https://github.com/nevans/resque-pool) -- which every Resque user should be using -- loads your process first, then forks the workers. Forking uses copy-on-write semantics, so as long as you're loading everything before forking, you're conserving a great deal of otherwise wasted memory on what would be shareable pages.