News for hackers. Stuff that matters.
Re:Polyglot is a distributed web framework for multiple prog... by floatboth on Monday July 14, 2014 @11:57AM (#8031494)
So, it's like mongrel2, but a bit worse: AMQP is centralized, unlike ZeroMQ.
Re:Polyglot is a distributed web framework for multiple prog... by tlrobinson on Monday July 14, 2014 @12:38PM (#8031741)
That was my thought as well, though I never really understood the point of Mongrel2.

It's basically a reverse proxy that speaks to upstream application servers using a custom protocol over ZeroMQ instead of HTTP over TCP? Why is this better than just using HTTP?

Does anyone here use Mongrel2? Do you like it?

Re:Polyglot is a distributed web framework for multiple prog... by rubiquity on Monday July 14, 2014 @01:58PM (#8032234)
I haven't used Mongrel2, though I have used ZeroMQ, but I can try to answer this question:

> Why is this better than just using HTTP?

In SOA most of your services aren't going to be exposed publicly. HTTP is a great protocol for public facing servers, but HTTP is a very clunky protocol. For private services, it's a pretty big benefit (performance, scalability and ease of parsing) to skip HTTP and use something else. ZeroMQ gives you several messaging patterns that you would never get from HTTP.

Re:Polyglot is a distributed web framework for multiple prog... by FooBarWidget on Monday July 14, 2014 @02:09PM (#8032289)
That makes sense if your services do not follow the request-response pattern (e.g. background workers). But what if they do? In Polyglot, the services very clearly follow a request/response pattern because they're handling web traffic. What sense, then, does it make to use a message queue?

Persistence? Makes no sense for web traffic. Even if the message is persisted to disk, it's useful for a few minutes at most before the user gives up and closes the tab.

Language-independence? You don't need a message queue for that. You can do that with regular HTTP.

Scaling and load balancing? Ditto.

Re:Polyglot is a distributed web framework for multiple prog... by rubiquity on Monday July 14, 2014 @02:14PM (#8032319)
I'm not sure what your comment is getting at. I think HTTP is fine for the Web, and any service exposed and designed for use by the Web is going to have to use HTTP. But if you're going to use a framework like Polyglot, you're going to have several services and any of those services not directly communicating with the Web doesn't need to speak HTTP.
Re:Polyglot is a distributed web framework for multiple prog... by FooBarWidget on Monday July 14, 2014 @02:32PM (#8032416)
What I'm getting at is why those services shouldn't speak HTTP. I understand that they don't need to speak HTTP per se, but I get the feeling that your comment is implying that such services should speak a non-HTTP protocol, while I think that it's fine even if those services speak HTTP.
Re:Polyglot is a distributed web framework for multiple prog... by socceroos on Tuesday July 15, 2014 @01:35AM (#8034956)
Persistence can be very important depending on your needs. Save-and-forget comes to mind.
Re:Polyglot is a distributed web framework for multiple prog... by tlrobinson on Monday July 14, 2014 @02:45PM (#8032500)
That makes sense. There are also good reasons to just HTTP: one less technology to understand/troubleshoot, reuse existing tools and middleware.
Re:Polyglot is a distributed web framework for multiple prog... by Xorlev on Monday July 14, 2014 @04:10PM (#8032923)
We've had a similar argument internally. In load testing, we're not bounded by the speed of our JSON parsing, and if we were we'd probably just swap that endpoint to use a different message body provider before moving to a totally binary RPC model.

Especially in the Java world, where Jackson + Afterburner is fast enough for most cases. Protobufs/Thrift will smoke it in most performance tests, true enough, but when you're waiting on a database or algorithm to run, what's JSON serialization?

There's certainly benefits at extreme scale, but not enough to justify the loss of tooling that comes with it until necessary.

Re:Polyglot is a distributed web framework for multiple prog... by rubiquity on Monday July 14, 2014 @02:51PM (#8032524)
Yup, I agree with that and in a team environment that may be very valuable. HTTP is definitely the "JavaScript of protocols." However, I don't think learning other protocols is a ton of overhead, and learning is always a good thing ;)
Re:Polyglot is a distributed web framework for multiple prog... by jkarneges on Monday July 14, 2014 @04:06PM (#8032896)
We use Mongrel2 at Fanout.io. I love it. It's part of our general ZeroMQ architecture of having lots of components that each do one thing well.

ZeroMQ is an improvement over plain HTTP primarily because you can use fewer internal pipes by interleaving requests and responses over the same sockets.

We even created a HTTP over ZeroMQ spec to help standardize this approach: http://rfc.zeromq.org/spec:33

Re:Polyglot is a distributed web framework for multiple prog... by FooBarWidget on Monday July 14, 2014 @06:37PM (#8033660)
What do you mean by internal pipes? Do you mean file descriptors? Why is reducing the number of file descriptors a good thing? After all, the total amount of traffic stays the same.
Re:Polyglot is a distributed web framework for multiple prog... by jkarneges on Tuesday July 15, 2014 @05:59PM (#8039336)
If you have a lot of fds, then you have fd polling issues to deal with for high performance.

To be honest I've not actually benchmarked anything comparing the throughput of an efficient fd poller to multiplexed pipe. It just feels nice to avoid the problem, considering what kind of effort can go into those pollers.

Some discussion from Zed about fd polling: https://web.archive.org/web/20120225022154/http://sheddingbi...

Re:Polyglot is a distributed web framework for multiple prog... by steveklabnik on Monday July 14, 2014 @03:58PM (#8032849)
This might be a weird edge case, but in Rust, 0MQ bindings came really quickly, as its protocol was easy. This let Rust do web stuff _far_ earlier than the still-ongoing work to actually write a server in Rust itself.

At least in Ruby world, we steal Mongrel's parser over and over and over and over. Seems like it might be easier to not have to do that, and just use Mongrel2 with a 0MQ library. I haven't actually done this, though...

Re:Polyglot is a distributed web framework for multiple prog... by CMCDragonkai on Monday July 14, 2014 @08:10PM (#8033988)
The advantage of Mongrel2 is this: http://zef.me/4502/message-queue-based-load-balancing

Message queue based load balancing.

Also dynamic load balancing without restarting any daemons.

Re:Polyglot is a distributed web framework for multiple prog... by kodablah on Monday July 14, 2014 @12:45PM (#8031782)
AMQP/Rabbit can use TLS (even if not leveraged in this project, something not possible for ZeroMQ IIRC). Also, define "centralized". An intelligent AMQP client properly falls over to another node in a cluster, and Rabbit offers good durability guarantees while still allowing quorum and tolerating node failure.
Re:Polyglot is a distributed web framework for multiple prog... by FooBarWidget on Monday July 14, 2014 @01:49PM (#8032176)
But we're talking about HTTP requests here. The RabbitMQ durability is useless for this use case. Suppose the RabbitMQ node fails, and the admin notices after 2 minutes and solves the problem after 2 more minutes. The user who initiated the HTTP request has long pressed Stop in his browser.
Re:Polyglot is a distributed web framework for multiple prog... by kjaleshire on Monday July 14, 2014 @02:16PM (#8032327)
True, durability wouldn't help here, however, individual Rabbit queues can fail over to alternate nodes if the master node for that queue stops responding.
Re:Polyglot is a distributed web framework for multiple prog... by calineczka on Monday July 14, 2014 @12:18PM (#8031618)
Exactly what I was wondering. How different from mongrel2 it is.