Re:River: A fast, robust job queue for Go and Postgres
by AtlasBarfed on Tuesday November 21, 2023 @06:45PM
(#38372302)
I thought the first rule of queues was never use a database as a queue.
Re:River: A fast, robust job queue for Go and Postgres
by 22c on Tuesday November 21, 2023 @08:30PM
(#38373391)
The "problem" is that a database as a queue doesn't infinitely scale, but nothing infinitely scales. If you're starting to reach the limits of Postgres-as-a-queue then you've either done something very wrong or very right to get to that point.
It's probably more correct to say that the engineering effort required to make a Postgres-as-a-queue scale horizontally is a lot more than the engineering effort required to make a dedicated queueing service scale horizontally. The trade-off is that you're typically going to have to start scaling horizontally much sooner with your dedicated queuing service than with a Postgres database.
The argument for Postgres-as-a-queue is that you might be able to get to market much quicker, which can be significantly more important than how well you can scale down the track.
Re:River: A fast, robust job queue for Go and Postgres
by AtlasBarfed on Wednesday November 22, 2023 @09:19AM
(#38379492)
I thought Kafka could relatively infinitely scale
It's particularly suited to use cases such background jobs, workflows or other operations which occur within your application and scales well enough for what 99.9999% of us will be doing.