Re:Job queues are deceptively tricky
by nachexnachex on Thursday July 16, 2026 @03:54PM
(#48939429)
Similar to what gp mentioned, max latency. You can tune your consumers to burn through the queue and push messages, but when you have latency constraints e.g. a very tight deadline the first few messages in the queue might achieve it but not the 20000th.
When you want to switch from no processing to say 2M messages than need to be sent within 5 minutes it's difficult to appropriately provision without fast scale out.
Iām skeptical. You can support a pretty massive messaging system with one box.
What did you try? What went wrong?