Re:System design hack: Postgres is a great pub/sub and job s...
by gigatexal on Friday November 08, 2019 @01:42PM
(#21485340)
The caveat to this is in the first paragraph: all of the data is stored in memory id each row is ~1kb and the table size has some sort of TTL you’ll be good.
[ Parent
| on HN ]
Re:System design hack: Postgres is a great pub/sub and job s...
by colinchartier on Friday November 08, 2019 @01:44PM
(#21485372)
Yeah, but even without tuning, you can have hundreds of "sparse" queues with this pattern - you don't have to care about the frequency of insertions or deletions or the size of the rows.
Re:System design hack: Postgres is a great pub/sub and job s...
by gigatexal on Friday November 08, 2019 @01:47PM
(#21485392)
I mean I agree I think with what I think you’re saying: as much as I love Postgres and SQL I’d much rather use Redis for this.
Re:System design hack: Postgres is a great pub/sub and job s...
by colinchartier on Friday November 08, 2019 @01:53PM
(#21485432)
I actually used redis pub/sub before switching to triggers - it is significantly easier to just:
INSERT INTO JOBS
versus INSERT INTO JOBS
redis-cli LPUSH job-queue job-id
It's also easier to configure, redis has BRPOPLPUSH for acked-queues, but it's much easier to just query jobs with a select statement from the database.
Re:System design hack: Postgres is a great pub/sub and job s...
by Rapzid on Friday November 08, 2019 @03:12PM
(#21486211)
Obviously "it depends" but here are a few reasons one may want to use PostgreSQL instead:
1.) *They want to transactionally commit work along with the change that caused it
2.) They are already using Postgresql not using Redis
3.) Requiring users install yet another service(Redis) just for this one item isn't worth the costs
Re:System design hack: Postgres is a great pub/sub and job s...
by gigatexal on Friday November 08, 2019 @03:36PM
(#21486452)
Redis modifications are transactional: https://redis.io/topics/transactions
Though I agree on points 2 and 3, especially 3 since it adds complexity.
Re:System design hack: Postgres is a great pub/sub and job s...
by erichocean on Saturday November 09, 2019 @03:36PM
(#21494185)
> Redis modifications are transactional
That's only true if they do not error—there is no "rollback" feature in Redis, and if you do error, anything you've done up to that point remains.
That's not transactional, it's more "you can, sometimes, execute a bunch of separate statement atomically".
Re:System design hack: Postgres is a great pub/sub and job s...
by gigatexal on Saturday November 09, 2019 @04:14PM
(#21494393)
I’ll admit I don’t know the internals of Redis as well as I do relational databases. I’ve learned much from this thread.