News for hackers. Stuff that matters.
Re:SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a... by throwaway5959 on Tuesday April 11, 2023 @12:47PM (#35527707)
> To make all of this run smoothly, we enqueue and dequeue thousands of jobs every day.

The scale isn't large enough for this to at all be a worry. The biggest worry here I imagine is ensuring that a job isn't processed by multiple workers, which they solve with features built into Postgres.

Usually I caution against using a database as a queue, but in this case it removes a piece of the architecture that they have to manage and they're clearly more comfortable with SQL than RabbitMQ so it sounds like a good call.

Re:SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a... by KrugerDunnings on Tuesday April 11, 2023 @02:01PM (#35528759)
It is easy to avoid multiple workers processing the same task: `delete from task where id = (select id from task for update skip locked limit 1) returning *;`
Re:SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a... by throwaway5959 on Tuesday April 11, 2023 @02:05PM (#35528825)
I didn't say it was difficult, I just said it was the biggest concern. That looks correct.