You know how people often bill sqlite as a replacement for (postgresql|mysql|oracle) but it is really best thought as a replacement for fopen()?

zeromq is the same way. It's not a replacement for something like kafka or rabbitmq, it's a replacement for socket()

Zed Shaw once called it "sockets that actually work how programmers think sockets work", but I can't find a link.

I love this description of zmq and sqlite.

This is a great way to put it.

Software is a huge domain. It’s just one of those things you’ll need when you need.

Some places you might use it: real-time financial data streams, multi-agent worker task/management, distributed database syncing. It’s basically a networking lib that makes common patterns easier. E.g send this message to everyone, list connected peers, handle these messages this way if the peer can’t take the message, and so on. They support doing this in a bunch of languages over a bunch of communication protocols.

Now all the Async/coroutine/ipc verbiage is mostly just because synchronization is a communication problem which is what zeroMQ happens to solve as well.

As OP points out, listing all connected peers is not supported.

ØMQ promises being "socket on steroids" but I used it quite a bit and it's not so beneficial.

It hides the scalability problem of having many TCP sockets without solving it (e.g. pub-sub over TCP).

It does not solve the ahead-of-line blocking issue.

It requires reading a lot of documentation and doing experiments to find out what is the right combination of " fan-out, pub-sub, task distribution, and request-reply" that you have to use.

e.g. https://zguide.zeromq.org/docs/chapter4/ https://zguide.zeromq.org/docs/chapter5/ https://zguide.zeromq.org/docs/chapter3/

Plus a lot of "magic" happening in the library involves queues that you have no control over. You get applications that hang and fill buffers instead of failing early.

I suspect this is the reason why it never became a new layer in the TCP/IP stack

The biggest limitation I found was the lack of reliable delivery semantics.

zmq is "a better socket()", but unlike sockets it's pretty hard to figure out what's going on. E.g. most (all?) socket types don't need the other side to be immediately available, it'll buffer a couple thousand or so messages and start blocking if it doesn't appear. From the app's point of view, everything seems fine until the queues are full (which you can't introspect, btw.) when suddenly everything starts blocking.

It does bring some nice things to the table, but using it has always felt at least a little bit "weird" to me. The docs are a little weird, and there are random caveats sprinkled around the library (e.g. you don't need to synchronize the startup of dependent services like described above, unless you were using certain transports in older versions and so on).

It used to be marketed somewhat towards developing multi-threaded applications, but it was always rather unclear what exactly the advantage of using, for example, the comparatively slow PUSH/PULL socket (ipc:// are pipes) over a MPMC (or as needed) queue ought to be.

It is also marketed towards resilient distributed apps, but, perhaps I was holding it wrong, but it'd always behave much like TCP and stuff like REQ/REP would pretty much always block forever if the peer goes away, while stuff like PUSH/PULL isn't reliable when peers go away, so for a distributed app these can't really be used anyway. So for this use case it always seemed rather too low-level and its guarantees too weak / non-applicable.

> REQ/REP would pretty much always block forever if the peer goes away

unless you tell it to do otherwise, that's exactly what it will do. The guide covers this: 4. Reliable Request-Reply Patterns

> It is also marketed towards resilient distributed apps, but, perhaps I was holding it wrong, but it'd always behave much like TCP and stuff like REQ/REP would pretty much always block forever if the peer goes away, while stuff like PUSH/PULL isn't reliable when peers go away,

I had exactly the same (disappointing) experience.

thank you. this is exactly how I feel about it.

zmq appears to be a layer that tries to hide some of the most important details that the socket layer exposes. socket programming and the state machines that tend to go along with it are some of the least difficult parts of systems programming to do well, it is well documented, well understood and there are ample tools to help measure ground truth. it was always weird to me that zmq had such a strong reputation; it basically hides what would otherwise be a few hundred lines of library code (including a framer).

You can think of ZeroMQ as legos for building message queue-based messaging systems. ZeroMQ gives you different pieces - pushers, pullers, publishers, subscribers, routers, etc. and it's up to you to then hook them up in different ways to create the system you want. The guide gives you some different ideas of how to hook up the different pieces for common use cases, but there's nothing stopping you from building a custom one.

ZeroMQ is not an out-of-the-box MQ solution. It takes some work to understand and build a functional MQ solution with ZeroMQ. If you want something that works out of the box I recommend RabbitMQ.

> You can think of ZeroMQ as legos for building message queue-based messaging systems.

Are there are any, preferably open source, message queue based messaging systems using ZeroMQ?

For internal machine-to-machine communications you can replace your full blown HTTP server + JSON inefficencies with a simple ZMQ request/reply + binary serialisation format.

It also allows one machine to broadcast a message to a 'topic' that other machines listen for.

Many other things. It's not perfect, but it Just Works.

From the site,

”ZeroMQ (also spelled ØMQ, 0MQ or ZMQ) is a high-performance asynchronous messaging library, aimed at use in distributed or concurrent applications. It provides a message queue, but unlike message-oriented middleware, a ZeroMQ system can run without a dedicated message broker.”

I read it as a low-level abstraction for when you want to build a custom messaging infrastructure.

This tells your “what” not “why”.

I prefer the “replacement for socket” pitch, personally.

I believe the 'problem' that it solves is universal network transparency between pieces of anything. It's a library that can be used from basically any language that has a C FFI on any system with a C compiler to do IPC with anything else supporting same using a uniform and relatively simple API.

Yeah, I've never used it myself, but I think the main value is an abstraction over programs talking to each other.

Like you can tell it to use sockets if the machines are remote, or different inter-process communication if they're running on the same machine, etc.

It also has some common patterns like pub/sub and others.

It's an alternative to writing your own complex networking code for any kind of distributed communications.

A lot of people these days might use something like flink or spark to build a pipeline whereas I'd prefer to write a number of small servers with various fan out or fan in operations, broadcast style operations, request/reply semantics etc where appropriate

Makes way more sense and much simpler to my mind and allows much better control over performance characteristics.

Mongrel 2 used zeromq to proxy to workers. I don't know how popular that ever was in practice.

For a long time I've wondered how applicable it could be as a potential replacement for dbus.