Minus the "natively supported" part, we have this today in http://www.gevent.org/ ! It's so, so empowering to be able to access the entire historical body of work of synchronous-I/O Python libraries, and with a single monkey patch cause every I/O operation, no matter how deep in the stack, to yield to your greenlet pool without code changes.
We fire up one process per core (gevent doesn't have good support for multiprocessing, but if you're relying on that, you're stuck on one machine anyways), spend perhaps 1 person-day a quarter dealing with its quirks, and in turn we never need to worry about the latencies of external services; our web servers and batch workers have throughput limited only by CPU and RAM, for which there's relatively little (though nonzero) overhead.
IMO Python should have leaned into official adoption of gevent. It may not beat asyncio in raw performance numbers because asyncio can rely on custom-built bytecode instructions, whereas gevent has "userspace" code that must execute upon every yield. And, as with asyncio, you have to be careful about CPU-intensive code that may prevent you from yielding. But it's perfect for most horizontal-scaling soft-realtime web-style use cases.
Then came Asyncio, which I personally disliked for the simple reason it became so popular that everybody thinks it's necessary to write an asyncio version of their library/module. The result is it now tears everything in 2. Asyncio code and non-asyncio code ...
Everything like Asyncio, Curio, Trio are impressive and interesting but their interfaces are way too involved for the intended result and it should never be like that. (Gevent suffered less from that).
The only way forward is proper thread support without the GIL and therefor I applaud this effort.
If you can solve the local case, you can solve the global case. Pick your technique; any ol' technique is just as good as any other.
> gevent doesn't have good support for multiprocessing
This is where my little library https://github.com/jgehrcke/gipc might help! It's been working well for people since 2012 :)
By the way, gevent is huge for the Python ecosystem and I would love to acknowledge both Denis Bilenko and Jason Madden who have done a great job over the years. I am in particular impressed by Jason's maintenance work -- creating a new project is always exciting. Maintaining it over the years is sometimes just tedious, hard work. Jason has impressed me a lot with his one man show there.
Not sure if everybody knows, but gevent was for starters built on top of "only" libev -- another super impressive project built and maintained by practically a single person: Marc Lehmann. I learned a lot from his work, too. So so so many serious software projects build on top of libev.
Only 'recently' gevent included support for libuv, the event loop that originated from the NodeJS ecosystem.
I find it super interesting to see that CPython+gevent+libuv is highly similar to running NodeJS!
By the way, the LWN article is once again a wonderfully written piece by Jonathan Corbet. His way to doing technical writing is simply great, I have enjoyed it very much over the years and learned a lot.
Are you able to fully utilize a multicore processor? I'm not familiar with gevent.
uWSGI supports gevent https://uwsgi-docs.readthedocs.io/en/latest/Gevent.html
Falcon also benefits from Cython acceleration. It's been a while but at the time I tested against PyPy and either it was slower or had quirks I was unable to resolve (don't recall w/o consulting my notes).
A far cry from the one-request-per-thread days of yore!
I'm wondering how would this work in a kubernetes/docker world.
The OP is trying to solve the GIL problem, which is saying: "activity on other threads, which might touch objects in memory that my thread cares about, shouldn't keep my current thread from doing useful work." Right now, Python needs to be super conservative to keep threads from stomping on each others' memory, and this should make it a lot more feasible to do that in a sane way. As the comments in the OP post itself suggest, though, this isn't necessarily a sufficient solution. If you're looking for the real solution to "let other threads borrow my memory without global locks," the Rust language and its borrow checker is really the holy grail there.
Regarding gevent exceptions, I find them actually quite natural to work with. If you're looking at old tutorials that talk about linking callbacks to handle exceptions, in practice I've never needed to do that - that stuff is primarily useful to those writing things like gunicorn itself. Pool.imap[_unordered], greenlet.get(), and more will simply re-raise exceptions in the caller's greenlet if there are exceptions in the farmed-out greenlets. And if you need even more control, you can try-catch within the functions that are running inside your greenlets, and return results that may either be a success or a failure.
Added: I don't understand the downvotes. If Python 3 was going to make an incompatible departure from Python 2, they might as well have done stuff like the above, that brought real benefits. Instead they had 10+ years of pain over relatively minor changes that arguably weren't all improvements.
I think it's important to remember the difference between parallelism (trying to compute faster by using multiple cpu's simultaneously) and concurrency (communicating on multiple communication channels that are sending stuff non-deterministically). The GIL gets in the way of parallelism but except in extreme cases, it doesn't interfere with concurrency.
It took less than a day to migrate all my code to Python 3. And by "less than a day" I mean "less than 2 hours". Granted, bigger projects would take longer, but saying stuff like "10+ years of pain" is ridiculous. Probably less than 1% of projects had serious issues with the migration. We just hear of a few popular ones that had some pain and assume that was representative.
I myself never bothered migrating any of my Python 2 stuff. It might not be difficult to do so, but continuing to run it under Python 2 still works fine. If you migrated all of yours in 2 hours, you must not have had much to start with.
I do use Python 3 for new stuff most of the time by now, but I keep running into little snags, like the .decode() method not working on strings, or having some (but not all) of the codecs removed from the string module so you have to use the codecs module.
There's also the matter of stuff that is supposedly ported but isn't completely. For example, Beautiful Soup works nicely under py3, but it doesn't have a typeshed entry so its import needs a special annotation to stop mypy from complaining about it.
The real loss with Python 3 is that it could have been so much better than it is. I remember hearing that Go expected to pick up a lot of migrating C++ users, but it got migrating Python users instead.
Here's a pain story about a 2 to 3 migration though:
https://dropbox.tech/application/how-we-rolled-out-one-of-th...
Thanks for all your work in the Python ecosystem!
I'm dubious about this work because I think it tries to paper over the cracks in code with data races, but maybe the problem was there already with the GIL and this changes little.
I agree about the annoyance of the codec modules, and stuff like the urllib reorg happening on 3.0 instead of shifting it to later releases (and self-owns like the u prefix). But if you're calling `decode` on a string you have a bug in hiding, from the moment that your "code that handles encodings transparently" blows up when someone gives you a file that isn't UTF-encoded.
signed: someone who did the 2 -> 3 transition for a project that deals with a lot of non-UTF-stuff.
Even the Python 2 strategy for hex might lead to your text getting garbled if you’re using anything not représentable in ASCII (roughly).
Python3 unnecessarily broke compatibility in many places. Yes I agree that using bytes.fromhex() and .hex() are better design choices. But requiring manual inspection and updating of all strong manipulation code for the transition was an outrageous requirement, particularly for a dynamic language that doesn’t (or at least didn’t at that time) have compile-time type checking to catch such errors.
Sorry, but that's fiction. Yes, lots of projects (still the minority) simply didn't upgrade, but it's not because they tried and failed. It's because they never prioritized it.
> Here's a pain story about a 2 to 3 migration though:
Consistent with my claim:
> Probably less than 1% of projects had serious issues with the migration. We just hear of a few popular ones that had some pain and assume that was representative.
Sure, you could hunt and find some that were still maintained/developed with Python 2 in, say, 2019. But these again will be the tiny minority.
I still stand by my 1% claim. The problems with upgrading have been vastly exaggerated.
It does seem in the spirit of Python's duck typing to be able to say '616263'.decode('hex') and get 'abc', and that does work fine in py2. Try it in py3 and you get a type error. So ok, convert the string to bytes, e.g. by saying b'616263' instead of '616263'. That doesn't work either, but I'll leave the experiment to you.
codecs.decode(b'616263', 'hex')
In fact, it's a perfectly viable strat in python to have several processes, each having several threads, each having an event loop.
And it will still be so, once this comes out. You will certainly use threads more, and processes less, but replacing 1000000 coroutines by 1000000 system threads is not necessarily the right strategy for your task. See nginx vs apache.
I really don't care that much which of them survive, I just want to rely on less of them
This is the same language that shipped with at least 3 different methods to apply functions across iterables when the Zen of Python was adopted as a PEP in 2004.
I'm not sure who is claiming that. Here's the OP we're replying to:
> They are techs with different trade off, and life is full of opportunities.
Zen would be: pick one of the two approaches, keep its strengths while fixing it to get rid of its weaknesses, then declare the fixed version as the one obvious way to do it. You might still have to keep the other one around for legacy support, but that's similar to the situation with applying functions across iterables.
This is what Go did. Go has one way to do concurrency (goroutines) and they are superior to both of Python's current approaches. Erlang has of course been in the background all along, doing something similar.
Besides, it's weird, like saying we should not have int, float and complex, there should be one way to do it.
Just because those are 3 numbers doesn't mean they don't have each their own specific benefit.
If you look at Haskell, Erlang/Elixir, and Go, they all let you write performant sequential code by pushing the async into the runtime where the programmer doesn't have to see it. Python had an opportunity to do the same, but stayed with async and coroutines. What a pain.
Well, that question is as old as engineering itself, and it's always a matter of resources, cost, history and knowledge.
With that said, sure, threads and asyncio are complimentary in the sense that you can run tasks on threadpool executors and treat them as if they were coroutines on an event loop. But that serves no purpose unless you're trying to do blocking IO without blocking your whole process.
I have used Trio in real projects and I thoroughly recommend it.
This blog post [2] by the creator of Trio explains some of the benefits of those libraries in a very readable way.
[1] https://trio.readthedocs.io/en/stable/
[2] https://vorpus.org/blog/some-thoughts-on-asynchronous-api-de...
Many more libraries use Asyncio than Trio, so I have come to recommend Anyio (which has a Trio-like API) but with Asyncio as the backend. That gives you the extensive Asyncio ecosystem but with the structured concurrency design of Trio.
But yes, consider alternatives before you pick asyncio as your approach!
Talk: https://www.youtube.com/watch?v=oLkfnc_UMcE
Blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
And since we are stuck with colored functions in python, the choice of stack matters very much.
Now, if you want easier concurrency, and a solution to a lot of concurrency problems that curio solves, while still being compatible with asyncio, use anyio:
https://anyio.readthedocs.io/en/stable/
It's a layer that works on top of asyncio, so it's compatible with all of it. But it features the nursery concept from Trio, which makes async programming so much simpler and safer.
Hypercorn works with trio..but you lose a LOT of performance
JavaScript doesn't need a GIL because it doesn't really have threads. WebWorkers are more akin to multiprocessing than threads in Python. Objects cannot be shared directly across WebWorkers so transferring data comes with the expense of serializing/deserializing at the boundary.
It isn't anymore.
In [3]: from asyncio import run
In [4]: async def async_func(): print('Ran async_func()')
In [5]: run(async_func())
Ran async_func()
In [1]: async def async_func(): print('Ran async_func()')
In [2]: await async_func()
Ran async_func()
Not sure it will get there, but it would be nice. I think putting a top level "await" is explicit enough for stating you want an event loop anyway.
Now, with TaskGroup in 3.11, things are going to get pretty nice, espacially if this top level await plays out, provided they include async for and async with in the mix.
Now, if they could just make so that coroutines are automatically schedules to the nearest task group, we would almost have something usable.
- The event loop in JS is invisible and implicit. V8 proved it can be done without paying a cost for it, and in fact most real life python projects are using uvloop because it's faster than asyncio default loop. JS dev don't think of the loop at all, because it's always been there. They don't have to chose a loop, or thinking about its lifecycle or scheduling. The API doesn't show the loop at all.
- Asynchronous functions in JS are scheduled automatically. On python, calling a coroutine function does...nothing. You have to either await it, or pass it to something like asyncio.create_task(). The later is not only verbose, it's not intuitive.
- Async JS functions can be called from sync functions transparently. It just returns a Promise after all, and you can use good old callbacks. Instantiating a Python coroutine does... nothing as we said. You need to schedule it AND await it. If you don't, it may or may not be executed. Which is why asyncio.gather() and co are to be used in python. Most people don't know that, and even if you know, it's verbose, and you can forget. All that, again, because using the event loop must be explicit. That's one thing TaskGroup from trio will help with in the next Python versions...
- the early asyncio API sucked. The new one is ok, asyncio.run() and create_task() with implicit loop is a huge improvement. But you better use 3.7 at least. And you have to think about all the options for awaiting: https://stackoverflow.com/questions/42231161/asyncio-gather-...
- asyncio tutorials and docs are not great, people have no idea how to use it. Since it's more complex, it compounds.
E.G, if you use await:
With node v14.8+:
await async_func(params)
import asyncio
async def main():
# no top level await, it must happen in a loop
await async_func(params)
asyncio.run(main) # explicit loop, but easy one thanks to 3.7
With node:
...
async_func(params)
...
# async_func(params) alone would do nothing
res = asyncio.create_task(async_func(params))
...
# you MAY get away with not using gather() or wait()
# but you also may get "coroutine is never awaited"
# RuntimeWarning: coroutine 'async_func' was never awaited
asyncio.gather(res)
Python is my favorite language, and I can live with the explicit loop, but explicit scheduling is ridiculous. Just run the damn coroutine, I'm not instantiating it for the beauty of it. If I want a lazy construct, I can always make a factory.
Now, thanks to the trio nursery concept, we will get TaskGroup in the next release (also you can already use them with anyio):
async with asyncio.TaskGroup() as tg:
tg.start_soon(async_func, params)
- no gather or wait. Schedule it, it will run or be cleaned up.
- no need to chose an awaiting strat, or learn about a 1000 things. This works for every cases. Wanna use it in a sync call ? Pass the tg reference in it.
- lifecycle is cleanly scoped, a real problem with a lot of async code (including in JS, where it doesn't have a clean solution)
You can't "just run the damn coroutine", that's not what coroutines are.
But that does point to the mistake of Python, at least from a UX perspective: coroutines are more efficient but they're also a lot more prone to use errors, especially in dynamically typed languages. Async functions should probably just have been tasks, which is what they are in JS.
It has been proposed to Python: https://groups.google.com/g/python-ideas/c/PN1_j7Md4j0/m/0xy...
But I have no more info about it.
await-ing that promise is often what you want to do but there are other ways to do work once the promise has resolved.
Well it has queues and threads already.
Its just that asyncio for socket handling at least (in the testing that I did) is about 5% faster. (one asyncio socket "server" vs ten threads [with a number of ways to monitor for new connections])
I always assumed that people wanted asyncio because they look at javascript and thought "hey I want GOTOs cosplaying as a fun paradigm"
Check out anyio if you want to use them now.
If you want to use structured concurrency now, IMO the best bet is to use Trio directly. Reading posts by the author makes it clear that every detail of the library had been extremely scrutinised, not just the API (e.g. see this long post on ctrl-C handling [3], or any number of long technical discussions on the issue tracker), so I think it's a better choice in any case.
[1] https://twitter.com/1st1/status/1041855365745455104
[2] https://docs.python.org/3.11/whatsnew/3.11.html
[3] https://vorpus.org/blog/control-c-handling-in-python-and-tri...
But it's asyncio compatible, which makes it more future proof.
Trio has the better API, so you might as well use that directly rather some other library that attempts to match it, even if it can/does use it under the hood. It reminds me of that xkcd about standards. It's also going to have to settle for the least common denominator to some extent even if it reimplements some parts itself (e.g. if one library implements serial ports but the other didn't, then it surely can't be in anyio). I also dispute that it's "more future proof", because you're still relying on anyio being maintained, and to me it seems to be the more obscure library.
I totally disagree with this negative perspective of asyncio.
async programming is MUCH easier to program and understand than other concurrency solutions like multithreading.
async works extremely well as a programming model as evidenced by the fact that it's exactly the model being adopted by all languages that want to implement practical and developer friendly concurrency.
When I read such a negative take on asyncio I assume only this is a developer who hasn't done alot of programming with it, and is therefore still somewhat lacking in full understanding.
If you do anything on the CPU or if you have any I/O which is not async you stall the event loop and everything grinds to a halt.
Imagine a program which needs to send heartbeats or data to a server in a short interval to show liveness, Kafka for example. Asyncio alone can't reliably do this, you need to take great care to not stall the event loop. You only have exactly one CPU core to work with, if you do work on the CPU you stall the event loop.
We see web frameworks built on asyncio but even simple API only applications constantly need to serialize data which is CPU-bound. These frameworks make no effort (and asyncio doesn't give us any tools) to protect the event loop from getting stalled by your code. They work great in simple benchmarks and for a few types of applications but you have to know the limits. And I feel that the general public does not know the limitations of asyncio, it wasn't made for building web frameworks on the async event loop. It was made for communicating with external services like databases and calling APIs.
But that's always going to be the case for single threaded code. If you're ready to use threads, then that integrates pretty seamlessly with asyncio by using an executor task.
> asyncio doesn't give us any tools to protect the event loop from getting stalled by your code
Again, check out executors: https://docs.python.org/3/library/asyncio-eventloop.html#asy...
Asyncio in Python and JS are very different beasts.
All those frameworks trade off high performance if everything is well behaved against easy of use and potential of very bad performance if some regression is introduced.
Whether that tradeoff ia a good one to take will depend on the application.
For nearly every popular library there is now a (usually inferior, less robust) async one. The benefits of Linus' Law are reduced.
Watch the very NSFW (lots of swearing) but hysterically funny video "node.js is bad ass rock star tech" on youtube sometime ;). https://www.youtube.com/watch?v=bzkRVzciAZg
This is almost always the case of async tasks disappearing into the void. After shooting myself in the foot with this one time to many I have a hard rule to never use create_task. Instead make sure every single task ends up in some kind of gather() or wait() awaited from the top level. This will ensure any exceptions are propagated. If the number of tasks are dynamically created, while others are still ongoing, add them to a common list and restart your wait() call.
Its not that strange actually. If you compare it to threads, where you shoot off a thread which you never join(). It’s exactly the same scenario.
Any thread which doesn’t have a join needs a watchdog. Or register the unhandled exception handler.
Can’t remember if threads do this by default. Asyncio tasks for sure don’t, and they can’t really do it either, since a task in error state could be saved for later use for legitimate reasons. Closest you get is the warning printed to log about tasks that got GCd without ever being awaited, if you enable ASYNCIO_DEBUG=1
IMO the reason that we have so many ways of doing concurrency in Python (fra asyncio, curio, trio to gevent, multiprocessing etc.) is that it was never properly dealt with.
It has to be built into the language, a feature of the language, like fx. garbage collection. The model of Pony or Erland, where are execution thread is started per core could also have been used in python, instead we got the async/await mess, which almost created a whole new language, where every library has to be rewritten.
It saddens me to think that the ugly cludge that async/await is got added to Python almost without any discussion, whereas the insignificant walrus operator got so much heat that GVR quit.
In an alternate Python timeline, asyncio was not introduced into the Python standard library, and instead we got a natively supported, robust, easy-to-use concurrency paradigm built around green/virtual threading that accommodates both IO and CPU bound work.