Yeah I'm a big fan of Good Job too.
As long as you stick to pure ActiveJob features it really shouldn't be too difficult to scale from in memory based queue to PostgresSQL backed Good Job and then to Redis backed Sidekiq if and only if you need to.
Re:Delayed Job vs. Sidekiq
by stephenhuey on Wednesday February 16, 2022 @11:33AM
(#30362089)
Yes, I wanted a PG-based one and reached for Good Job on a new project since it was recently created and with Active Job in mind from the start. Check out the intro:
The problem with postgres-backed queues is they can't be used with connection pooling, so your workers are unnecessarily using up DB connections.
Re:Delayed Job vs. Sidekiq
by waffle_maniac on Wednesday February 16, 2022 @09:50AM
(#30360653)
I thought that was an ActiveRecord problem. We use MYSQL and have a lot of long-lived connections.
Not sure how it works with MySQL, but with postgres the most common pooling options use transactional mode, which doesn't support LISTEN/NOTIFY.
rocket job is pretty solid too, has a lot of the pro sidekiq features for free https://rocketjob.io/
Re:Delayed Job vs. Sidekiq
by waffle_maniac on Wednesday February 16, 2022 @09:48AM
(#30360618)
Without the pro version it's not reliable:
> If a Sidekiq process crashes while processing a job, that job is lost. [0]
We use Sidekiq Pro at work but I find it unfortunate that reliability is a pro feature. It's really hard to tell a product owner: Yeah, we might lose a few jobs once in a while.
Re:Delayed Job vs. Sidekiq
by aaronbrethorst on Wednesday February 16, 2022 @11:30AM
(#30362068)
Seems like a good reason to pay Mike.
agree. its not a lot of money, or it didn't used to be last time I looked. If your org relies heavily on it just pay
Re:Delayed Job vs. Sidekiq
by aaronbrethorst on Wednesday February 16, 2022 @02:31PM
(#30364597)
I just looked this up. It's $995/year, or approximately 4% of the fully loaded cost of an FTE software engineer at most companies. i.e. about 3 weeks of a developer's time.
Re:Delayed Job vs. Sidekiq
by waffle_maniac on Wednesday February 16, 2022 @03:28PM
(#30365247)
I understand paying extra for niche features. Isn't reliability a core feature of background job software?
Re:Delayed Job vs. Sidekiq
by waffle_maniac on Wednesday February 16, 2022 @03:30PM
(#30365270)
We do pay for it.
Perhaps you could address my point? Open source background job software that isn't reliable by default? That makes sense?
However, this recent addition to the space seems to be gaining traction and appears to have an excellent feature set -
https://github.com/bensheldon/good_job