Re:Overhead of Python Asyncio Tasks
by mrnonchalant on Wednesday March 08, 2023 @03:24PM
(#35074269)
Thanks for this!
Re:Overhead of Python Asyncio Tasks
by jonathan_s on Wednesday March 08, 2023 @04:00PM
(#35074648)
If you can, best is to always spawn them in a task group (either using anyio or Python 3.11's task groups).
This prevents tasks from being garbage collected, but also prevents situations where components can create tasks that outlive their own lifetime. Plus, it's a saner approach when dealing with exception handling and cancellation.
Perhaps I just don't get them, but task groups never really made sense to me.
The whole beauty of async tasks is that you can spawn, retry, and consume them lazily. When you create a task group, you again end up waiting on a single long-running last task, desperately trying to fix individual failures and retries that hold up the entire group.
Here's why task groups may be a good idea
https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
Be careful to hold your references, because async tasks without active references will be garbage collected. I've been bitten by that in the past.
Long discussion here: https://bugs.python.org/issue21163
Docs: https://docs.python.org/3/library/asyncio-task.html#asyncio....
"Important
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done."