We looked into AMQP based solutions. Our understanding is that slaving, master-slave failover, or other strategies are used to mitigate the fact that there is a broker responsible for a given stream of messages.

We wanted a brokerless, decentralized, and distributed system. NSQ has no broker or SPOF so it is able to route around failures.

That said, I think RabbitMQ is a good tool depending on your requirements. I can imagine a broker proving useful in situations where you may want strict ordering or no-duplication. Those were tradeoffs we were willing to make for fault tolerance and availability.

Also, given the fact that we were already operating infrastructure built on simplequeue (which is also distributed and brokerless) we found it more appealing to evolve rather than replace.

A broker is an implementation detail; there's nothing stopping you from going peer-to-peer with $protocol and having publishers manage their own subscribers. Some sort of registry is still needed though; nslookupd in your case.

There is a bit of an impedance mismatch since now you have to go out-of-band to fetch information. Not saying it's a bad thing; that's essentially also the role of DNS and LDAP, among others, so it is a fairly common pattern.

Master-slave etc are HA techniques for replicating state eg queues (or "channels" as nsq calls them?). "Decentralized and distributed" has nothing to do with state, master/slave or any of that; it's a topological property of your system. You can certainly build a decentralized and distributed system using a set of brokers, regardless of whether they support HA.

That's fair, you certainly can use rabbitmq in a distributed and decentralized fashion.

You bring up an important point though... What is important is the topology we're promoting and the platform we've built to support it. The actual queue is less important and it made sense for us to own that piece to achieve our goals.

So, why not just deliver that topology using off the shelf components? Rabbit, Kafka, ZeroMQ, ... the toolbox is deep. This is not meant as a criticism, I am simply trying to understand what I am missing when I look at your design :-)