SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a Postgres Queue

Re:SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a... by valenterry on Wednesday April 12, 2023 @04:22AM (#35536528)

Because a queuing system offers a different thing than a (relational) database.

You can build a queuing system with a database, but you have to do that. Some of the features and constraints of the database might even make your life harder than it has to be.

Instead, view it like that: there is a need for a queuing system and a job system. Either or both can be implemented using a database for certain concersn, but it can also be a custom implementation. It's not a great idea to mix the two things unless the operational and infrastructure costs and complexity outweigh the benefits of a clear separation.

[ Parent | on HN ]

Re:SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a... by nicoburns on Wednesday April 12, 2023 @07:05AM (#35537743)

There are libraries that implement queueing on top of databases that require very little setup by the user. For example https://github.com/timgit/pg-boss

[ Parent | on HN ]

Re:SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a... by scrollaway on Wednesday April 12, 2023 @10:06AM (#35539871)

I’ve had a positive experience with Procrastinate.

https://procrastinate.readthedocs.io/en/stable/

[ Parent | on HN ]

Re:SQL Maxis: Why We Ditched RabbitMQ and Replaced It with a... by valenterry on Wednesday April 12, 2023 @10:39PM (#35550199)

Right. However, please don't forget that there are very often inherit limitations regarding scaling or availability and also that many fully fledged message queues come with a lot of perks like access management and administration / debugging tooling and interfaces.

I'm not saying that libraries like pg-boss and co. cannot sometimes replace a full queue implementation. But the tradeoffs need to be clear.

[ Parent | on HN ]