Message queues and message logs do different things. The idea of the log is that subscribers can show up after the log is written and read or reread it from the beginning. In an event sourced architecture you use the log as the source of truth and all consumers can replay the log against a local store to reconstruct a view of the system’s state. You also can use a log for pubsub, but if that’s all you need one of the MQ solutions is usually a better fit.

> Why use RabbitMQ and Kafka if you can use ZeroMQ?

They are totally different, you're comparing apples with oranges.

ZeroMQ gives you basic, very fast tooling to communicate between distributed processes. ZeroMQ does not provide tooling for e.g. maintaining a strictly ordered, multi-terabyte event log. And so on.

Yes but isn’t this a bit like comparing git / bitkeeper vs subversion / perforce?

Basically, one is decentralized and you can set up a massively parallel architecture, with eg each topic or subthread having its own pubsub.

The other is a monolithic centralized pubsub architecture.

You could argue that git in large institutional projects converges to a monolithic repo so at that point it’s less efficient even than svn.

But for most use cases, ZeroMQ would allow far more flexible distributed systems topologies and solutions. No?

Edit: HN and Google are both awesome: https://news.ycombinator.com/item?id=9634925

Zeromq is just a bit of sugar on top of tcp sockets. It isn't a message queue or anything close. You would be wasting a ton of time reimplementing a lot of basic features like retries, persistence, service discovery, dead letter queues, priority, and a ton of other stuff.

> You could argue that git in large institutional projects converges to a monolithic repo so at that point it’s less efficient even than svn.

Not true. Facebook and Google do not use Git. Microsoft does not use vanilla Git for their monorepo. They created this extension to make it scalable https://en.wikipedia.org/wiki/Virtual_File_System_for_Git