News for hackers. Stuff that matters.

A comment on Octobot - A Low-Latency, Highly Parallel Distributed Task Queue Worker. Parent comment

Re:Octobot - A Low-Latency, Highly Parallel Distributed Task... by DEinspanjer on Monday September 20, 2010 @03:55PM (#1710297)
Sorry for my confusion, so is the problem you experienced one of having thousands of queues that a small number of clients pulled from, or thousands of workers that worked off of a small number of queues?

In our case, we want to be having a very small number of queues that have a high rate of i/o with dozens of reader clients and writer clients.

[ Parent | on HN ]
Re:Octobot - A Low-Latency, Highly Parallel Distributed Task... by bananaandapple on Monday September 20, 2010 @06:20PM (#1710701)
Could you eloborate where you did have problems? We also plan on using a small number of queues having a high rate of readers and clients (e.g. distributed task execution).
[ Parent | on HN ]
Re:Octobot - A Low-Latency, Highly Parallel Distributed Task... by cnlwsu on Monday September 20, 2010 @11:08PM (#1711329)
We have a large number of queues with small number of workers.
[ Parent | on HN ]