phintjens
Explicit identities are dangerous for servers since they create entropy (state accumulation) unless (and even) if you set strict high-water marks.
They're also a real pain internally and as part of simplifying 0MQ's construction we eventually want to get rid of them.
There are better ways to keep and synchronize state among a set of subscribers. See the detailed Clone pattern in Ch5 of the Guide, for example.
http://zguide.zeromq.org/page:all#A-Shared-Key-Value-Cache-C...
1. The 0MQ devs originally got asserts backwards, using them to validate external input (e.g. on sockets) instead of internal consistency. We've been fixing this for a year or two now, and it's pretty good. You'll get assertion failures if you e.g. use sockets from multiple threads. Not so much if you pass bad stuff onto sockets.
2. 2.1 was a great step forwards, and the use of "sleep" was in toy examples. Real networking apps tend to run forever, so this message loss at exit wasn't a big deal. You're right that the product is still young.
3. Totally agreed, this lack of peer presence detection is annoying, and the source of some debate on the lists.
4. Threading model works fine for me, I've used it extensively. A usable reactor is a hundred lines of code, no more. See the libzapi zloop reactor, in C, for example.
5. Demultiplexing sounds like useful functionality but should probable sit above sockets.
Seriously, 0MQ has made network programming fun (again) in a bad, addictive, way. Any design I can think of turns into real working code in a few hours, sometimes days. And I'm using C, a language that isn't normally fun to work in.
Right now, it's multithreaded clients and servers for resilient shared distributed hash maps. Tomorrow, network-wide logging. After that, another message broker. And so on.
Yes, it's a negative experience. I'd like my old lazy life back.
For the love of god, don't try it.
Sorry for the comic book style. It's how my mind works. Feel free to send a patch. :)
The benefits of 0MQ to me, as a programmer, are:
* Really easy to write utterly solid multithreading code, in a language (C) that has zero support for this; * Really easy to take this same code and make it multiprocess, or multibox, with few changes. * Built-in handling of asynchronous I/O, which I need in any real application. * Speed, meaning I can be lazy and my code still runs very quicky. This is less obvious than it seems. 0MQ's critical path is extremely evil.
It's so hard to explain this with a diagram or a slideshow. The only way to understand is to use it, to write code with 0MQ. Certainly for me, when I started programming 0MQ apps, my understanding of what this library was about, and what it could do for me, totally changed.
The FOSDEM streaming server seems to be timing out right now.
Enlightenment requires that you change in some way. For me, like many people who have taken the step of downloading and learning 0MQ, the feeling of "wow, this is too easy, where's the catch?" comes only after a few days writing code, and then your brain makes a slight adjustment, and it's all obvious, and that is a small but real enlightenment.
To some of the comments here...
- Parallel programming may be taught in CompSci courses, for sure, and it's a popular use for 0MQ, but (a) it's not applied to mainstream application development, and (b) it is not designed to scale to networks of any size.
- The Actor model is not key to building a successful message-based concurrent application, but it's a good example, and helps people understand that there are alternatives to shared-state concurrency.
- It's IMO useless to ask people to learn Erlang or Scala to get message based concurrency. People use Java, C, C#, PHP, Python, and probably still COBOL somewhere. So the challenge is how to give this mainstream a toolset that lets them build large parallel applications.
Perhaps next year at FOSDEM we can do a devroom and take the time to see a real application evolve from a simple stand-alone process into a real distributed one. 0MQ is very hard to grasp as a theory, one needs to actually use it for a few days before the beauty a cheap, universal, fast, intelligent, easy to use, and asynchronous queuing messaging fabric hits home.
Cheers!
Try it, you'll be surprised, I guarantee it.
What JMS achieved was to somehow turn these two very different semantics into a single one, based around "destinations". That was very clever but also makes JMS weirdly complex to use because the queue (request-reply) pattern and the topic (publish-subscribe) pattern just don't work the same way. We designed AMQP originally by taking the JMS spec and reverse engineering that into a wire level protocol. Then around version 0.3 we threw out destinations and came up with a generic wiring model based on exchanges, bindings, queues. That was my summer holiday in 2005.
AMQP is BTW still some way away from a 1.0 standard and mainly it's been five years trying to get reliability working in the center of the network. That always seemed destined to failure, as I explained in a presentation to the working group in 2006: http://www.zeromq.org/whitepapers:switch-or-broker.
AMQP and JMS both focus on resources held in the center of the network. It's familiar to anyone using HTTP as their stack. Thin stupid clients talking to big smart servers. 0MQ turns the focus to smart clients working across a thin, stupid (but massive) network.
Both approaches have their value. We (iMatix and the 0MQ community) tend to believe we can do a lot more, faster and more cheaply, using the distributed approach.
As a programmer, using 0MQ is Better than Sex. You can quote me on that. It would be unethical to deny such fun to anyone. What beginners need, with any new technology, is a set of clear recipes that they can implement without having to invent everything. This was why a books like Stevens' trilogy were so vital to getting TCP out of labs and into the grubby hands of ADHD bleeding-edge types.
We're working on that, with books like http://www.zeromq.org/docs:user-guide. That'll grow to cover all of 0MQ eventually.
This lets us do things like start clients and THEN start a server... the clients automatically connect when the server comes along. The server can go away, be replaced by another on the same endpoint, and clients will gracefully and invisibly start talking to the new server.
In some cases this is precisely what we want, in other cases it's not. If we need to detect dead peers, we add heartbeating as a message flow on top of 0MQ. Most larger 0MQ applications currently do this. Eventually 0MQ sockets may offer heartbeating, it seems a natural evolution. ("Seems" but is not necessarily.)
Additionally there are some patterns (like synchronous request-reply) that simply don't handle network issues properly. If your service dies while processing a request, your client if it does a blocking read will hang forever. There are work-arounds such as using non-blocking reads.
It would be a mistake to try to solve all challenges at once in any project. 0MQ is taking it step by step, starting with the core questions of how to design massively-scalable patterns correctly. Just doing that is worth gold, as you can see from people actually using 0MQ, who consistently tell us, "this makes our life orders of magnitude easier, thank you".
And as we solve the core problems properly, we'll continue to address other aspects, either in 0MQ core or in layers on top of that, and this process will continue ad infinitum.
Just kidding. It's always nice to see science in action. Great work! I suspect there's an impact on ZeroMQ's own poll/epoll strategy.
We do intend to deliver IETF-quality specs that turn messaging patterns like pubsub into 1st class citizens of the Internet, so that arbitrary vendors can produce pieces that plug into this. But these specs will still be really simple.
I've always advocated a standard API for AMQP and proposed one (WireAPI) years ago, because this reduces vendor capture. Imagine if vendors designed arbitrary socket APIs... well some do, and it locks their users in.
So 0MQ is partly an API standardization project, partly an architecture project (to build a layer that seems to be missing from the stack), and partly a protocol specification project.
This is the metaprogramming we used to build OpenAMQ. It was very successful in technology terms (1M lines of real code generated from 20k lines of metacode) but a failure in social tems (too high a barrier to community participation).
We've waited 6 years for AMQP to develop a community and address basic issues such as its complexity and high cost of change. No progress... despite years (literally) of work to try to open the AMQP workgroup to wider participation.
We love the idea of open messaging protocols and invested a painfully huge amount in AMQP but it's not been worthwhile: almost as soon as we helped found the AMQP working group, the protocol was hijacked into something complex and untenable.
At iMatix we believe in simple, clean, fast software. Just run any AMQP broker and compare to 0MQ and the conclusion is clear: 0MQ wins, hands down.
0MQ still has a long way to go but it's totally open, and anyone with a good idea is welcome to contribute. No contracts to sign. Just join the mailing list and discuss.
We will make messaging patterns into 1st class citizens of the Internet.