News for hackers. Stuff that matters.
Re:Three quick tips from two years with Celery by simonpantzare on Wednesday July 29, 2015 @02:26PM (#9970576)
-Ofair disables task prefetching which will affect throughput negatively if most of your tasks run for a short time (http://celery.readthedocs.org/en/latest/userguide/optimizing...).
Re:Three quick tips from two years with Celery by mlissner on Wednesday July 29, 2015 @03:58PM (#9971186)
yeah, I don't get this suggestion at all. They say that:

> This option comes with a coordination penalty, but results in a much more predictable behavior

In general, I'd much rather the greater throughput over predictable behavior. In general, I'm not watching my queue and having it be predictable isn't a very high priority.

Re:Three quick tips from two years with Celery by taylorhughes on Wednesday July 29, 2015 @04:59PM (#9971527)
Ofair will increase throughput in a queue where you have mixed-length tasks, because in a default configuration long tasks will block the queue from starting new tasks. I believe this is because the main process distributes some number of tasks, waits for them to finish, then distributes another set of tasks, and so on. (Would love more clarity on this — I haven't dug into it too much.)

Assuming most Celery task queue work involves network requests that can be unpredictable, I think Ofair is a more sensible/understandable default.