Re:An Introduction to Solid Queue for Ruby on Rails
by ryeguy on Sunday May 11, 2025 @11:56AM
(#43954639)
How does this reduce operational overhead? You still need a queue and a worker to dispatch the tasks to your api.
Re:An Introduction to Solid Queue for Ruby on Rails
by latchkey on Sunday May 11, 2025 @11:59AM
(#43954669)
The end user doesn't need to implement the queue or the worker. They just need to implement the http api for receiving POST messages. The queue "server" is effectively just a http client that reads messages from the database and sends them off.
Re:An Introduction to Solid Queue for Ruby on Rails
by ryeguy on Sunday May 11, 2025 @08:10PM
(#43958388)
That makes sense as a general pattern, but probably less so in the context of rails which is typically a self contained monolith. It would be adding another hop, more indirection, and more complexity. It would introduce new problems like the need to segment your "real" api from your worker api for the purpose of load isolation.
Re:An Introduction to Solid Queue for Ruby on Rails
by latchkey on Sunday May 11, 2025 @08:14PM
(#43958401)
If it is a self contained monolith, that's perfect for this. You have a long running process, which scans a database for new work and then posts to localhost.
Re:An Introduction to Solid Queue for Ruby on Rails
by pythonaut_16 on Sunday May 11, 2025 @11:41AM
(#43954536)
Why would GCP Cloud Tasks be part of the default libraries? If itβs that simple surely a community maintained gem would be fine?
Re:An Introduction to Solid Queue for Ruby on Rails
by latchkey on Sunday May 11, 2025 @11:58AM
(#43954658)
To clarify: I'm talking about implementing the core functionality of GCP Cloud Tasks, not using that specific product.
Re:An Introduction to Solid Queue for Ruby on Rails
by ARama on Sunday May 11, 2025 @04:31PM
(#43956899)
We use to do this by making use of the cloudtasker gem.
However we've recently just migrated to Sidekiq. We regularly found that the time to enqueue a job with cloud tasks was anywhere from 200-800ms. We often have workflows from users where they need do something that would involve a batch of say 100+ jobs. The time to process each job was minuscule, but actually enqueuing them took a long time.
Time to enqueue with Sidekiq / redis was brought down to under 20ms. We can also make use of bulk enqueuing and enqueue all 100+ jobs in one redis call.
However when we were first starting out, it was a godsend. Made our infrastructure significantly more easy to manage and not have to worry.
Since this is part of Rails, all you would need to do is implement regular http endpoints, no need for workers/listeners. Submit a "job" to the queue (which itself is just a POST) and the message details: the endpoint and some data to POST to said endpoint.
The queue "server" processes the list of jobs, hits the specified endpoint and when it gets a 200 response, it deletes it. Otherwise, it just keeps retrying.
[0] https://cloud.google.com/tasks/docs