News for hackers. Stuff that matters.
Re:ZeroMQ: Modern & Fast Networking Stack by mahmud on Friday September 03, 2010 @03:11PM (#1660714)
ZeroMQ is quickly becoming an even bigger hammer in the premature optimization planet of Newbo-Thumbia.

Edit:

1) It's a networking library; no admin tools or other soft handle-bars, like user-space utilities.

2) It uses a binary protocol. Good luck debugging that with syslog.

It's a very powerful tool in the hands of a capable systems architect, who actually needs it. For the rest, it's pretty much like an adult male tiger; excellent to watch in its natural habitat from a safe distance, terrible pet idea for you and your fiancee (and not because you live in a studio apartment.)

Re:ZeroMQ: Modern & Fast Networking Stack by dtf on Saturday September 04, 2010 @02:58AM (#1662215)
It's got absolutely nothing to do with optimization, it just happens to be rather fast. As far as I know, there is know other tool out there, fast or slow, that offers zeromq's level of simplicity.

1) It's a networking library

Yes, but one that's easy to use. Versus BSD sockets, which have an annoying array of quirks when programming for multiple platforms.

Moreover, it's consistent whatever your transport. One thing I've found using zeromq is that, because it's so simple to use, it makes you want to think in terms of message passing. I use a queue to push messages between threads at practically no cost. Then later if I feel like it I can shunt the consumer to a different process, or a different host, or several different hosts with a load balancer. I could do the same with raw sockets, but I'd rather eat my own leg with a spoon.

2) It uses a binary protocol. Good luck debugging that with syslog.

It's just a packet. You can put whatever you want in it: XML, JSON, protocol buffers, msgpack, JPEG.... The wire protocol is trivial.

It also has jolly nice bindings for Python, Ruby, Lua etc, allowing you to bridge between C or C++ and your scripted components with zero hassle.

Go on, give it a go.

Re:ZeroMQ: Modern & Fast Networking Stack by aaronblohowiak on Sunday September 05, 2010 @04:55AM (#1664244)
"As far as I know, there is know other tool out there, fast or slow, that offers zeromq's level of simplicity."

STOMP! Stomp is simpler than 0mq, but you have a central server instead of point to point.

Re:ZeroMQ: Modern & Fast Networking Stack by sustrik on Sunday September 05, 2010 @07:01AM (#1664337)
Not really. Compare 0MQ's wire protocol:

http://rfc.zeromq.org/spec:2

and STOMP:

http://stomp.codehaus.org/Protocol

Re:ZeroMQ: Modern & Fast Networking Stack by bretthoerner on Friday September 03, 2010 @03:58PM (#1660901)
Is it? I thought this was a level of abstraction higher? The premature optimizer would want access to the raw sockets this is built ontop of. I could be misunderstanding, though, I've never used it. The idea I have in my head is,

ZeroMQ: Send a message, other side either receives full message as a single payload (or nothing at all? or an error is raised? I'm not tototally sure).

Sockets: write(), read() and friends. Other side has to make sure he has the whole message, otherwise read() again and wait, deal with errors on its own, etc.

Re:ZeroMQ: Modern & Fast Networking Stack by mahmud on Friday September 03, 2010 @04:14PM (#1660965)
I have seen higher-level libraries than Zero, I have seen middle-ware with better persistence than Zero, I have seen networking libraries better integrated with their languages than Zero. But I haven't seen a faster beast.

It's right there in their <title/>: "Fastest. Messaging. Ever".

The great majority of the people would just benefit from learning socket programming better, and sticking to something native to their tongue. Use the messaging infrastructure that integrates best with your development process, or opt for something with more padding, like Rabbit.

Call me weak, but I like infrastructure software whose guts I can peek, lsof, cat, grep, log, strace and diff. Stuff that I can take a snapshot of and reproduce. Network services should be transparent to the application code base. I wanna poke a whole in my firewall, give some hint to inetd, a few lines of cron and be done with it. I simply don't trust my own software enough to be sure I will not screw up the infrastructure.

Re:ZeroMQ: Modern & Fast Networking Stack by bretthoerner on Friday September 03, 2010 @04:25PM (#1661008)
I completely agree with the desire to be able to use Unix As It Should Be.

That said I am a little envious when I read that I can send("bunch of shit") and receive() and call it a day. I wouldn't call that premature optimization, nothing is going to beat raw sockets and domain specific knowledge.

That said, since you brought up AMQP (we use Rabbit, too, I'm a fan) have you seen this?: http://lists.zeromq.org/pipermail/openamq-dev/2010-March/001...

Re:ZeroMQ: Modern & Fast Networking Stack by mahmud on Friday September 03, 2010 @04:47PM (#1661102)
First, function level semantics and brevity are not exactly differentiators when most anyone can wrap the socket API with the class/parametric/duck-typed interface of their choice (what is github if not a pile of DWIM wrappers.) See how Twisted did wonders for network programming at the semantic level.

Also, I have already stated that it was a powerful tool in the hands of a "capable systems architect". Besides, I am a closet iMatix fan ;-)

http://news.ycombinator.com/item?id=1230509

All I wanted to point out is that ZeroMQ should not be treated as another "plugin" that people are encouraged to use willy nilly. I would use it in high-performance applications in financial trading, VOIP, scalable game servers, etc.

But these are not what most people are working on. Blog posts like these just encourage the ADHD bleeding-edge types to take another stab at their own thighs in the unjustified pursuit of "web scale". And the moment SmashingMag gets a hold of Zero, we will see the most gruesome newbie bloodshed since the battle of WordPress.

Re:ZeroMQ: Modern & Fast Networking Stack by phintjens on Sunday September 05, 2010 @02:43PM (#1664917)
Lol. Actually I believe 0MQ (or rather, the principles it embodies) is something all serious developers need to learn. How to do concurrency properly, how to get above TCP sockets, how to design simple APIs, how to do open source properly, etc. It's hard to see that exposure to 0MQ would make anyone a worse developer.

As a programmer, using 0MQ is Better than Sex. You can quote me on that. It would be unethical to deny such fun to anyone. What beginners need, with any new technology, is a set of clear recipes that they can implement without having to invent everything. This was why a books like Stevens' trilogy were so vital to getting TCP out of labs and into the grubby hands of ADHD bleeding-edge types.

We're working on that, with books like http://www.zeromq.org/docs:user-guide. That'll grow to cover all of 0MQ eventually.

Re:ZeroMQ: Modern & Fast Networking Stack by sustrik on Sunday September 05, 2010 @02:11AM (#1664111)
Speaking of AMQP/RabbitMQ there's an integration project here: http://github.com/rabbitmq/rmq-0mq. The idea is that you can use RabbitMQ as a broker for 0MQ networks.
Re:ZeroMQ: Modern & Fast Networking Stack by sandGorgon on Saturday September 04, 2010 @04:44AM (#1662330)
suppose you wanted to build a private-messaging system for a website, where users can send PMs to each other. Would zeromq be a good candidate for that (as the backend).

Would you need to use a broker for that ?

Re:ZeroMQ: Modern & Fast Networking Stack by mahmud on Saturday September 04, 2010 @05:48AM (#1662403)
No, it wouldn't be a good choice because it's not persistent in the physical, written-to-disk sense. It's an in-memory queue.

And no, you don't need a "broker" for that. The messaging middleware application of ZeroMQ, and other middleware, is not the same "messaging", in the human communication sense. The former can be used to implement the later, but typically it's for inter-application messaging and communication.

In your case, I recommend you use a tradition RDBMs and do something like

  CREATE TABLE Message
  (
      messageId   SERIAL,
      senderId    VARCHAR(20) NOT NULL,
      recipientId VARCHAR(20) NOT NULL,
      timestamp   TIMESTAMP DEFAULT NOW(),
      message     TEXT,
      status      message_status,           //
      PRIMARY KEY (messageId)
  );


   CREATE SEQUENCE serial;
   CREATE TYPE message_status AS ENUM ('draft', 'sent', 'new', 'read', 'archived', 'deleted');
Then a message "send" is just a row insertion. To refresh the display, in semi realtime fashion, you can have some javascript on the client side poll your server periodically, assuming you don't have comet or some other persistent connection.

If you have a persistent connection, then the code that inserts to the database also puts out a ticket in some "refresh-display queue" in the following manner. All logged in users are eligible for message status updates. i.e. those that have an active session will get "you have new email" message. You will have an alerter process/thread/job that runs in the background and reads those message update tickets. It takes the recipient ID and updates that user's display, i.e. pushes.

Typically, you would use ZeroMQ in just that later part; as a queue of updated-message statuses, and the queue is consumed by a background job that just notifies a bunch of remote users that they have new updates. Javascript on the client side takes care of the display update/modification (typically a DOM operation, though you could also have a flash or java applet consume pushed data; those later two can actually keep persistent connections, but watch out for session length or you might quickly run into file descriptor lossage.)

Re:ZeroMQ: Modern & Fast Networking Stack by aaronblohowiak on Saturday September 04, 2010 @03:05AM (#1662220)
The easy deployment, easy snooping and administrative tools are all big wins for STOMP on ActiveMQ, even if it doesn't hit millions of messages a second.