Re:The Big Little Guide to Message Queues
by candiddevmike on Monday January 29, 2024 @08:04PM
(#39185056)
Pub/Sub is meant to follow the Kafka "insane amounts of data" firehose pattern. Everyone thinks they need that scale unfortunately and skips over Cloud Tasks.
The Kafka pattern isn't about the size of your data, it's about accommodating heterogeneous consumers and easy fan-out. I don't love the ampq exchange model because it adds state to the middle of the queue.
Re:The Big Little Guide to Message Queues
by xyzzy_plugh on Monday January 29, 2024 @07:52PM
(#39184944)
Pub/Sub is truly the stuff of nightmares compared to Tasks. I wish Tasks were a little more loved, because I keep ending up using Pub/Sub inevitably.
What's funny is Pub/Sub is a fundamentally different model from message queues: queues are normally tightly coupled, meaning you enqueue a message to be processed by a specific system. Pub/Sub on the other hand is fundamentally loosely coupled: the publisher doesn't care who (if anyone) gets the message.
(I've heard "orchestration" vs "choreography" to describe this dichotomy, but I can't say I'm a fan of the jargon.)