News for hackers. Stuff that matters.
Re:Celery 2.3 released by jws on Friday August 05, 2011 @11:37AM (#2850409)
What is Celery?

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

The execution units, called tasks, are executed concurrently on a single or more worker servers using multiprocessing, Eventlet, or gevent. Tasks can execute asynchronously (in the background) or synchronously (wait until ready).

Re:Celery 2.3 released by StavrosK on Friday August 05, 2011 @01:06PM (#2850771)
Ask described it to me as "a library for discovering bugs in the multiprocessing library".
Re:Celery 2.3 released by asksol on Friday August 05, 2011 @02:03PM (#2851014)
Yes, but note that multiprocessing rocks. It's just that celery uses it so extensively :) (and now also eventlet/gevent if that suits better)
Re:Celery 2.3 released by jnoller on Friday August 05, 2011 @02:23PM (#2851096)
And someone here has commit privs to fix those bugs too :) (ask does)
Re:Celery 2.3 released by StavrosK on Friday August 05, 2011 @05:44PM (#2851915)
I agree with you on both counts. I've used multiprocessing to great effect, and, although it was fantastic for what it did, it was very buggy...

Nowadays, I'd just use Celery instead!

Re:Celery 2.3 released by reinhardt on Friday August 05, 2011 @03:24PM (#2851348)
Does Celery support task dependencies? Would be great to specify a DAG of tasks and have them run in parallel. Basically something like Oozie but not restricted to Hadoop.
Re:Celery 2.3 released by dangoldin on Thursday August 11, 2011 @11:09PM (#2875825)
If I understand your question correctly, you should be able to do this using callbacks.