Re:Multi-Tenant Queues in Postgres
by klysm on Friday April 19, 2024 @04:40AM
(#40084562)
Using Kafka as a work queue is widely documented as a mistake. Using a single database results in a lot of operational simplicity and you get to skip a lot of distributed systems when all your state is in a single system
[ Parent
| on HN ]
> Using Kafka as a work queue is widely documented as a mistake
Could not find any good sources on this. Can you please provide any references?
don’t have any off the top of my head sorry, I could be making it up but I’m pretty sure I remember reading that from several sources. Iirc it has to do with the fact that there isn’t really single message acknowledging?
Re:Multi-Tenant Queues in Postgres
by perfectspiral on Saturday April 20, 2024 @12:11PM
(#40098397)
This is a proposal to extend Kafka to (better) handle queueing use cases: https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A...
"For example, a queue is perfect for a situation in which messages are independent work items that can be processed concurrently by a pool of applications, and individually retried or acknowledged as processing completes. This is much easier to achieve using a queue rather than a partitioned topic with a consumer group."