In Python, thread pools and async work mostly the same in practice, except that threads have a bigger overhead. So they can be used to solve the same problems.
With Python and asyncio, neither assumption holds, but the idea of shaving off some latency persists: if we rewrite everything using asyncio, our DB accesses can be parallelized, yay! This may or may not be a net win in latency; reworking your DB access patterns may gain you more.
Nope: https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a...
But often the lower-hanging fruit is doing more joins in the DB, fetching fewer columns, and writing your query more thoughtfully. Async only helps if you can do something while waiting for the DB to complete your query. A common anti-pattern, exacerbated by ORMs, is doing a bunch of small queries while passing bits of data between them in Python; moving that to asyncio won't help much, if any.
Reluctant to add a "me-too" comment but I think you have it there. For whatever reason people are much keener to investigate different ways of scheduling IO than they are to investigate what the query plan looks like and why.
Really? I definitely had a huge struggle reading async python, because generators and yields break the concept of "what is a function" and struggled writing async python because of function coloring.
To me this violates a core principle: the principle of least surprise. It completely changes how I have to reason about function execution, especially if there are conditionals and different yields.
You can write web servers in a sync manner; Flask, Django is way better if you only need an API, choosing FastAPI for a simple REST API is a huge mistake.
FastAPI can be sync or async, although sync will have "a small performance penalty"
https://github.com/tiangolo/fastapi/issues/260#issuecomment-...
The only thing async io gives you is scalability of many MANY concurrent io operations. You have to either be pushing seriously large traffic or doing a lot of long running websocket/sse/long polling type requests.
The only other use case is lots of truly concurrent io within one request/response cycle. But again that is unusual, most apis have low single digit db queries that are usually dependent on one another removing any advantage of async.
Why just "within one cycle"? What about situation where you spend 1s in single db query and are 100ms cpu bound during request handling?
The point is asyncio is to have find grade control of yealding.
https://christophergs.com/python/2021/06/16/python-flask-fas...
maybe the right aproach would have been a Threadpool? Plus you don't have to refractor the task. Just let it run sync, but you can make it also async
I'm not very experienced with Kotlin, but that sounds similar to python's run_in_executor [1]
[1] https://docs.python.org/3/library/asyncio-eventloop.html#asy...
In Python there's no benefit in doing this due to the GIL, unless you're using a module which implements its own multithreading (for example in C). Python is not alone with this issue.
In other languages you're basically stepping out of the async paradigm in order to use threading in parallel. You can wait for result of the thread in the async loop without blocking.
I really enjoy using asyncio in Python for things where I have to do a lot of stuff in parallel, like executing remote scripts in a dozen of servers in parallel via AsyncSSH or for low workload servers which query databases.
In any case, what keeps me hooked on Python like a junkie is the `reload(module)` which I invoke via an inotify hook every time a file/module changes:
server.py
server_handler.py
reloader.py
server.py loads reloader.py (which sets up inotify) and that one takes care of reloading server_handler.py whenever it changes. server.py defers all the request/response handling to server_handler.py which can be edited on the fly so that the next request executes the new code. Instant hot reloading.
This also works very nice with asyncio (like aiohttp) and one ends up with very readable code.
If you need performance, then use Java, Rust, Go or C/C++, but for prototyping and tooling I absolutely love this approach with Python.
There you also have to act differently based the task
The docs on that page also point you to a different task library which does what I'd expect:
"If the work is appropriate for concurrency and parallelism, also consider using the Task Parallel Library."
I checked those docs and that is a framework that makes sense.
You can amplify this; pure Python is not meant for CPU-bound tasks. If you have a pure Python task, you've CPU optimized it somewhat, and it's still too slow, the answer is, get out of pure Python.
There's half-a-dozen options that still leave you essentially in Python land (just not pure Python anymore) like cython or NumPy, and of course dozens of other languages that leave Python entirely.
To accelerate a pure Python task to the speed that you can get out of a single thread of a more efficient language, you have to perfectly parallelize your task across upwards of 40 CPUs (conservatively!), because that's how slow pure Python is. asyncio isn't a solution to this, but neither is any sort of multiprocessing of the pure Python either.
This is not criticism. This is just engineering reality. Pure Python is a fun language, but a very slow one.
The value is allowing you to do something else while waiting for IO to finish. If you are constantly doing IO, async is a good tool. Otherwise, it won’t help.
I've used curio as an alternative concurrent programming engine for a med-large project (and many small ones). In comparison to asyncio, it's a joy to use.
The overhead to do a similar service using threadpools is much higher and the reason asyncio exists.
You wouldn't do it that way for any reason other than you can get better performance than way than by using threading or other pre-emptive multitasking.
It's not a better developer experience than writing straight forward code where threads are linear and interruptions happen transparently to the flow of the code. There are foot guns everywhere, with long running tasks that don't yield or places where there are hidden blocking actions.
It reminds me a bit of the old system 6 mac cooperative multitasking. It was fine, and significantly faster because your program would only yield when you let it do it, so critical sections coule be guaranteed to not context shift. However, you could bring the entire machine to a halt by holding down the mouse button, as eventually an event handler would get stuck waiting for mouse up.
Pre-emptive multitasking was a huge step forward -- it made things a bit slower on average, but the tail latency was greatly improved, because all the processes were guaranteed at least some slice of the machine.
"Scalability" is a better word to use when talking about asyncio. Along with describing complex concurrent programming such as for a GUI, where the added syntactical complexity if outweighed by the reduced boilerplate of traditional GUI programming.
Just this week, I got to the bottom of a performance issue in django because the developers were using async, and then doing eleventy billion db queries to import a big csv, thereby blocking all other requests.
One of the questions I ask on our programming interviews is "how would you make this (cpu bound) thing go faster" -- Async is definitely a low quality answer to that, when things like indexing or hash lookup vs array scan are still on the table.
I'd need to know the methodology. Asyncio is "gated", meaning that coroutines execute in tranches. When you queue up a bunch of coroutines from another coroutine, they don't execute right away instead they go in a list. Then when the current tranche completes the next one starts. There's some tidying up which occurs with every tranche.
As an (only one) example of "measuring the wrong thing", if you queue up a single task 1000 times which takes as its argument a unique timer instance it matters whether you queue them up one at a time as they finish running or queue them all up at once.
#!/usr/bin/python3
# (c) 2022 Fred Morris, Tacoma WA USA. Apache 2.0 license.
"""Illustrating the tranche effect in asyncio."""
from time import time
import asyncio
N = 1000
class TimerInstance(object):
def __init__(self, accumulator):
self.accumulator = accumulator
self.start = time()
def stop(self):
self.accumulator.cumulative += time() - self.start
return
class Timer(object):
def __init__(self):
self.cumulative = 0.0
return
def timer(self):
return TimerInstance(self)
async def a_task(timer):
timer.stop()
return
def main():
loop = asyncio.get_event_loop()
timing = Timer()
overall = time()
for i in range(N):
loop.run_until_complete( loop.create_task( a_task(timing.timer()) ) )
print('Sequential: {} Overall: {}'.format(timing.cumulative, time() - overall))
timing = Timer()
overall = time()
for i in range(N):
loop.create_task( a_task(timing.timer()) )
loop.stop()
loop.run_forever()
print('Tranche: {} Overall: {}'.format(timing.cumulative, time() - overall))
if __name__ == '__main__':
main()
# ./tranche-demo.py
Sequential: 0.02229022979736328 Overall: 0.04146838188171387
Tranche: 6.084041595458984 Overall: 0.012317180633544922I either use multiprocessing, or celery. using async/await looks ugly, doesn't solve most problems I would want it to, and feels unstable.
Maybe once development around it settles down I'll revisit, but I can't imagine wanting to use it in it's current form.
And yeah, async programming (in Python) isn't really for CPU bound stuff. You might benefit from multiprocessing and just use asyncio to coordinate, which is what it excels at. PyPy can really help with CPU bound stuff too, if the code is mostly pure Python.
So they are operating outside of the request without slowing it down.
We had used aync since 2014 ( twisted, tornado), and officially having async await in python starting 3.7 was the best thing happened to python.
Also nobody mentions this for some reason, but asyncio doesn't make your programs faster, in fact it makes everything 100x SLOWER (we measured it multiple times, compared the same thing to the sync version), but makes your program concurrent (NOT parallel), where the tradeoff is you use more CPU, maybe less memory, and can saturate you bandwidth better. Never do anything CPU-bound in an async thread!