News for hackers. Stuff that matters.
Re:Choose Postgres Queue Technology by lmm on Sunday September 24, 2023 @10:40PM (#37639125)
> Skype used postgres as queue with a small plugin to process all their CDR many years ago. I have no idea if it used these days but it was 'web scale', 10 years ago. Just working, while people on the internet argued about using a database as a queue is an anti-pattern.

It works great until it doesn't, and the way it breaks puts you in a state that's very difficult to recover from. And if your excuse for using a database as a queue was that you were already running a database, that cuts both ways: congratulations, your queue mess has now brought down your primary datastore too.

Re:Choose Postgres Queue Technology by dventimi on Sunday September 24, 2023 @11:01PM (#37639255)
I wonder if it would be possible to have more than just one PostgreSQL database.
Re:Choose Postgres Queue Technology by lmm on Sunday September 24, 2023 @11:19PM (#37639352)
It is, but it comes with a lot of the same costs as having a PostgreSQL database and a proper queue system.
Re:Choose Postgres Queue Technology by mianos on Sunday September 24, 2023 @11:23PM (#37639374)
No, you already know how to run and manager a postgres database.

If have you large teams, for sure, pick a proper queue that someone in the team knows well.

Re:Choose Postgres Queue Technology by lmm on Sunday September 24, 2023 @11:27PM (#37639395)
> No, you already know how to run and manager a postgres database.

Well, maybe. Postgres is large and has lots of features. NOTIFY/LISTEN is a distinct thing with its own quirks, and just because you've been running a postgres database up until now doesn't mean you're going to know about the admin side of these other features.

Re:Choose Postgres Queue Technology by dventimi on Monday September 25, 2023 @12:36AM (#37639715)
Now, you're making a different argument.
Re:Choose Postgres Queue Technology by hn_throwaway_99 on Sunday September 24, 2023 @11:12PM (#37639312)
> congratulations, your queue mess has now brought down your primary datastore too.

Just don't put your queue tables/logic in the same DB instance as your datastore. There are still a lot of benefits to using the same tech even if you have segregated instances.

Re:Choose Postgres Queue Technology by mianos on Sunday September 24, 2023 @11:22PM (#37639364)
> It works great until it doesn't, and the way it breaks puts you in a state that's very difficult to recover from.

This is similar to saying, 'if I mess up all the tables in one database I wreck the rest'. Just my opinion, but this is not actually a thing in databases.

Maybe compromised the performance of one database due to another loading things up? I think database are developed with this as an important consideration. I can't say I have seen this, but admit it's a possibility.

Also, if you run one postgres, you won't have of an issue running another if you have the experience in production.

Re:Choose Postgres Queue Technology by lmm on Sunday September 24, 2023 @11:28PM (#37639401)
> This is similar to saying, 'if I mess up all the tables in one database I wreck the rest'. Just my opinion, but this is not actually a thing in databases.

If you mess up the tables in one database it doesn't affect others, but if you lock up the server where it can't respond to queries, that affects every database running on that server.

> Also, if you run one postgres, you won't have of an issue running another if you have the experience in production.

We're talking about using a different feature that you presumably haven't used before, so you won't necessarily know about the admin side of that.

Re:Choose Postgres Queue Technology by j16sdiz on Monday September 25, 2023 @01:09AM (#37639845)
> If you mess up the tables in one database it doesn't affect others, but if you lock up the server where it can't respond to queries, that affects every database running on that server.

How is it different from: putting multiple queues on same redis, when one queue is locked up, others queue are affected?

If that's a real risk, you can always put them into different instances. The solution is exactly the same for redis or postgresql

Re:Choose Postgres Queue Technology by lmm on Monday September 25, 2023 @06:31PM (#37651627)
> How is it different from: putting multiple queues on same redis, when one queue is locked up, others queue are affected?

Queues by nature tend to be for tasks that you can tolerate delaying a bit. If your queues can impact your "live"/online processing, that's worse than just impacting other queues.

Also something like redis tends to be a lot simpler and less prone to locking up than the monster that is postgresql.