Re:River: A fast, robust job queue for Go and Postgres
by qaq on Monday November 20, 2023 @12:26PM
(#38351041)
Postgres based job queues work fine if you have say 10K transaction per second and jobs on average do not take significant time to complete (things will run fine on fairly modest instance). They also give guarantees that traditional job queues do not.
[ Parent
| on HN ]
Re:River: A fast, robust job queue for Go and Postgres
by Rapzid on Tuesday November 21, 2023 @03:04AM
(#38360757)
Probably order of magnitude more or perhaps a multiple of that depending on the hardware and design.
In theory an append-only and/or HOT strategy leaning on Postgres just ripping through moderate sized in-mem lists could be incredibly fast. Design would be more complicated and perhaps use case dependent but I bet could be done.
Re:River: A fast, robust job queue for Go and Postgres
by qaq on Tuesday November 21, 2023 @09:25AM
(#38364004)
Yep that's why I specifically mentioned "fairly modest instance" on reasonably fast box you can get magnitude more. You can partition the tasks table to reduce the number of rows skip locked has to run through to grab next task.