News for hackers. Stuff that matters.
Re:Scaling with Queues by mnutt on Tuesday September 10, 2013 @04:51PM (#6363399)
What happens when your RabbitMQ queue goes down and messages start building up in redis? Won't that eventually use up all of the memory on the machine?

We followed a similar path using Redis as a queue, and ended up having to write a 'valve' process that watches the queue and starts popping off messages into a file when the queue length gets too long. At that point we had half-reimplemented a real queue.

Re:Scaling with Queues by bhaisaab on Tuesday September 10, 2013 @05:21PM (#6363591)
So you make sure that you've multiple RabbitMQ servers or a RabbitMQ cluster so there is no single point of failure :)
Re:Scaling with Queues by mnutt on Wednesday September 11, 2013 @12:06AM (#6365460)
Ok, more simply, what about a network partition where the redis machine gets cut off from the RabbitMQ servers?

My only point is really just that redis-as-a-queue has some bad failure modes if you aren't careful.

Re:Scaling with Queues by bhaisaab on Wednesday September 11, 2013 @02:52AM (#6365887)
Sure, I agree and I understand things go wrong, so we've monitoring tools (munin, pingdom, pagerduty etc.) and we check them often or they post notifications often. There are at least two folks on-call 24x7. Based on present workloads we've calculations on how much time it would take to exhaust resources and we plan our servers accordingly, this buys us time to react.