News for hackers. Stuff that matters.
Re:My RabbitMQ setup for notifications by rickhanlonii on Friday July 26, 2013 @03:23PM (#6109977)
There's a cool and important feature to this setup that makes RabbitMQ specifically a great choice: queuing one message sends the notification to all devices at once.

In RabbitMQ, messages are published to exchanges and based on the type of the exchange, messages will be routed to specific queues, which is where the messages are consumed.

There are three types of exchanges you can publish to: Direct, Fanout, and Topic. In a Direct exchange (which is the default), a queue subscribes to the exchange with a routing key and any messages delivered to the exchange with that routing key is delivered to that queue to be consumed. So in a sense it's 1-1: exchange -> key -> queue.

However, in a fanout exchange, every queue subscribed to an exchange gets a copy of the message. This means that when you deliver a message to a fanout exchange, that exchange will give a copy of that message to every queue that is listening.

So you can send the "New Item" message to the "notify" fanout exchange, and every queue bound to that exchange will get the "New Item" message; i.e. the phone notification service, the web client notification service, etc, will all get the "New Item" message and can react accordingly. What makes this even better, is that if you add a new service (e.g. a new iPhone app), all you have to do to tap into your notification system to receive new item alerts is to tell a queue to listen to the notify exchange! You hear me right: you don't have to change any existing code.

Now, Andrei Marcu's aamnotifs is written in Python and he uses pika as his Rabbitmq module. Pika is a little more relaxed about the exchange types: any queues subscribed to an exchange with the same routing key will be treated as if they are attached to a fanout exchange, and will all receive copies of each message sent to the exchange with that routing key.

I say all of this just to point out the important point that Andrei left out: to add a second, third, or nth listener to your notification queue which receives a copy of the updates, you just have to add another

    n.receive("routing_name", print_notification)  
line and change print_notification to whatever method you want the message to go to. For example:

    import notifs

    def print_notification(title, message):
        print "Notification received: {0}: {1}".format(title, message)

    def web_app_notify(title, message):
        print "Webapp notification received: {0}: {1}".format(title, message)

    def iphone_app_notify(title, message):
        print "iPhone App notification received: {0}: {1}".format(title, message)

    def android_app_notify(title, message):
        print "Android App notification received: {0}: {1}".format(title, message)

    try:
        n = notifs.Notifs("amqps://user:[email protected]:5673/%2F")
        n.receive("routing_name", print_notification)
        n.receive("routing_name", web_app_notify)
        n.receive("routing_name", iphone_app_notify)
        n.receive("routing_name", android_app_notifify)

    except KeyboardInterrupt:
        break
gist here: https://gist.github.com/rickhanlonii/6091542
Re:My RabbitMQ setup for notifications by eterm on Friday July 26, 2013 @03:35PM (#6110051)
Is it possible to have it so that it's fanout but a "die on first receive"?

i.e. If I send out a notification I don't care if it's my phone, my desktop or my laptop that gets the notification, but if one of those devices gets the message (race conditions not withstanding) then I don't want the other devices to get the message.

Or is it the case that if they weren't online they'd not be currently subscribed so they wouldn't naturally get it? In which case what happens if there are no current subscribers?

Re:My RabbitMQ setup for notifications by rickhanlonii on Friday July 26, 2013 @06:08PM (#6110880)
You can use a direct queue for this. Just have the phone, desktop, laptop, et. al. consume from the same queue. Rabbit gives out messages in a round-robin to the listening consumers, and the first to pickup the message is the only queue to get the message.

Then, let's say there's an error in the phone notification and you want to pass it back and let the next service try. This is where Rabbit's ACK system can come into play: you can setup rabbit to wait for an ACK response from a consumer acknowledging that the message was received and processed. If the consumer errors, you can return the message back to Rabbit and it will be given to the next consumer in line (which is the next consumer not busy).

If a service is not online, then they won't be consuming from the queue, so they won't get the message. If there are no consumers, then the message will wait in the queue until a consumer binds to consume the message.

Hope this makes sense!

Re:My RabbitMQ setup for notifications by jeffasinger on Friday July 26, 2013 @03:49PM (#6110148)
You could share one queue amongst several devices, and Rabbit will ensure that only one device gets the message (with proper acking on the subscriber side).

If a message gets sent to an exchange that has no queues attached, typically it will get sent to a dead letter exchange.

Edit: Most Rabbit client libraries have two different modes, one where you automatically acknowledge any message that is received, and another mode where you must explicitly acknowledge that you've received the message.

Re:My RabbitMQ setup for notifications by silasb on Friday July 26, 2013 @03:47PM (#6110136)
I was wondering about this also. I'm not sure if there is an acknowledgement system built into RabbitMQ, but it sounds like that is what would be needed for this to happen.
Re:My RabbitMQ setup for notifications by rickhanlonii on Friday July 26, 2013 @06:10PM (#6110894)
By default any message picked up from a queue by a consumer is removed (acknowledged), but you can configure the exchange so that it waits for an ACK from the consumer before removing the message. This gives the consumer the ability to return the message to the queue, but is dangerous because Rabbit will never stop waiting on the consumer to respond.
Re:My RabbitMQ setup for notifications by old_sound on Saturday July 27, 2013 @04:01AM (#6112188)
Pika is a little more relaxed about the exchange types: any queues subscribed to an exchange with the same routing key will be treated as if they are attached to a fanout exchange, and will all receive copies of each message sent to the exchange with that routing key.

Actually that's how RabbitMQ works. The direct exchange will send messages to every queue bound to it if the routing key matches. As in: "SELECT q FROM queues where q.binding_key = msg.routing_key"

The fanout exchange will do the same but without the WHERE clause.

The topic exchange will do as the direct exchange but let's say that instead of an equals comparison it will do a match to accommodate routing key patterns for topics.

Of course RabbitMQ doesn't use SQL, this is just for illustrative purposes.

Re:My RabbitMQ setup for notifications by jeffasinger on Friday July 26, 2013 @03:57PM (#6110208)
Routing with Rabbit is extremely flexible, in fact, you can hook up two exchanges together (with a Rabbit only extension). You can also give messages a TTL, so that they'll automatically delete if not fetched.

So you can have one exchange for each user, and any device the user owns attaches a queue to that. Then, any exchange that a user is interested in only needs to subscribe to that one exchange.

Re:My RabbitMQ setup for notifications by rickhanlonii on Friday July 26, 2013 @06:15PM (#6110916)
Rabbit really is awesome. Even on top of that you can setup reply-to queues so that a publisher can wait for a response from the consumer. This is nice if you have, say, an image processing server and want to give it an image to process and block until you get a response that it's done.
Re:My RabbitMQ setup for notifications by andreimarcu on Friday July 26, 2013 @03:43PM (#6110108)
Indeed, I had the impression that this will not be as clear, I'm going to edit my post to mention it.

Thank you!