News for hackers. Stuff that matters.
Re:System design hack: Postgres is a great pub/sub and job s... by colinchartier on Friday November 08, 2019 @01:44PM (#21485372)
Yeah, but even without tuning, you can have hundreds of "sparse" queues with this pattern - you don't have to care about the frequency of insertions or deletions or the size of the rows.
Re:System design hack: Postgres is a great pub/sub and job s... by gigatexal on Friday November 08, 2019 @01:47PM (#21485392)
I mean I agree I think with what I think you’re saying: as much as I love Postgres and SQL I’d much rather use Redis for this.
Re:System design hack: Postgres is a great pub/sub and job s... by colinchartier on Friday November 08, 2019 @01:53PM (#21485432)
I actually used redis pub/sub before switching to triggers - it is significantly easier to just:

  INSERT INTO JOBS
versus

  INSERT INTO JOBS
  redis-cli LPUSH job-queue job-id
It's also easier to configure, redis has BRPOPLPUSH for acked-queues, but it's much easier to just query jobs with a select statement from the database.
Re:System design hack: Postgres is a great pub/sub and job s... by Rapzid on Friday November 08, 2019 @03:12PM (#21486211)
Obviously "it depends" but here are a few reasons one may want to use PostgreSQL instead:

1.) *They want to transactionally commit work along with the change that caused it

2.) They are already using Postgresql not using Redis

3.) Requiring users install yet another service(Redis) just for this one item isn't worth the costs

Re:System design hack: Postgres is a great pub/sub and job s... by gigatexal on Friday November 08, 2019 @03:36PM (#21486452)
Redis modifications are transactional: https://redis.io/topics/transactions

Though I agree on points 2 and 3, especially 3 since it adds complexity.

Re:System design hack: Postgres is a great pub/sub and job s... by erichocean on Saturday November 09, 2019 @03:36PM (#21494185)
> Redis modifications are transactional

That's only true if they do not error—there is no "rollback" feature in Redis, and if you do error, anything you've done up to that point remains.

That's not transactional, it's more "you can, sometimes, execute a bunch of separate statement atomically".

Re:System design hack: Postgres is a great pub/sub and job s... by gigatexal on Saturday November 09, 2019 @04:14PM (#21494393)
I’ll admit I don’t know the internals of Redis as well as I do relational databases. I’ve learned much from this thread.
Re:System design hack: Postgres is a great pub/sub and job s... by Rapzid on Friday November 08, 2019 @03:52PM (#21486604)
Internally but if the change causing the work is in PostgreSQL Redis would need to support 2-phase commits to commit the change and the work together atomically(and durably).