An introduction to RabbitMQ

Re:An introduction to RabbitMQ by kabacha on Friday May 22, 2020 @12:54AM (#23268757)

I'm using a bit of a hyperbole but using redis list as a task queue is a simple as:

    while True:
        msg = r.lpop(key)
        try:
            result = do_something(msg)
        except Exception as e:
            log.error(f'failure for msg "{msg}" got {e} back to queue {key}')
            r.rpush(key, msg)
            continue

Pop a member from a list, if failed plop it back to the end. It's simple, explicit and just works™

[ Parent | on HN ]

Re:An introduction to RabbitMQ by lbruder on Friday May 22, 2020 @04:29AM (#23269843)

It just works, until PUSHing the member back fails. At least use RPOPLPUSH and a separate worker queue to make sure you don't accidentally drop packets.

Don't get me wrong, I'm using Redis queues myself and trying to get rid of the last remnants of RabbitMQ in our code, but there are use cases where RabbitMQ means less thinking about stuff, because it just works(tm)...

[ Parent | on HN ]

Re:An introduction to RabbitMQ by jlouis on Friday May 22, 2020 @05:49AM (#23270287)

The other classic variant here is when msg is poisonous. Then pushing it back onto the queue means it'll go again at some point, poisoning your system again.

In practice, you need some dead-lettering with a message timeout. Something RabbitMQ does provide out of the box, for all its complexity.

[ Parent | on HN ]

Re:An introduction to RabbitMQ by ayush--s on Friday May 22, 2020 @05:46AM (#23270266)

couple of problems with this: (1) RPUSH can fail (2) LPOP's result might never reach your client after redis has executed it.

[ Parent | on HN ]