Re:Celery, non-blocking code and quest against coroutines
by garethrees on Thursday June 09, 2016 @10:54AM
(#11869766)
You're not addressing the point about shared data. How do I program my Celery tasks to operate safely on shared data?
Well, I need a system that is responsible for maintaining the shared data, and I have to program my tasks to send it queries and updates, and receive the results, and so probably I'm going to need serializers and deserializers for all my data structures, and if the tasks have transactions (which they almost certainly do) then I'm going to have to implement locks to maintain the data integrity. Whatever it is, this system looks a lot like a database.
Whereas with coroutines each task just updates ordinary data structures in memory. No need for communication, queries, serialization, or locks.
Re:Celery, non-blocking code and quest against coroutines
by sunkencity on Thursday June 09, 2016 @10:43AM
(#11869683)
well GCD is an evented model and Celery is not so what's your point?
Queue and Thread is available in python and work well up to a point. Twisted (and sugar @inlineCallbacks) or similar makes for much more concise code.
And it's not that I think Celery is the best. It has huge drawbacks.
I merely suggesting that the model Celery and GCD enforce is much better for writing CPU, I/O and CPU+I/O bound code.