News for hackers. Stuff that matters.
Re:The Python Steering Council intent to approve PEP 703: ma... by JonChesterfield on Saturday July 29, 2023 @03:14AM (#36917722)
Exciting. Python is mostly written as C shared libraries that knew they had a global lock to rely on.

Some of those do sufficiently simple things that they can run without any locking and all will be fine.

Others will still need locking, but are now under pressure to run without the gil. Some of those are going to do DIY locking within their own bounds.

Maybe what python has really been missing all these years is loads of ad hoc mutex calls scattered across the ecosystem. Data races and deadlocks introduced in the name of performance is not how I expected python to go out.

edit: expanding on this pessimism a bit.

Making C libraries written assuming a global lock thread safe is the sort of thing I'd expect concurrency experts to advise against and then make mistakes while implementing it. My working theory is that most people who wrote C extensions for python are not concurrency experts and are great programmers who won't back down from a challenge.

The data-race/hang/segfault consequences of this combination look totally inevitable to me. Python application developers are not going to love the new experience and I'm thankful my products are not built on top of the python ecosystem.

Re:The Python Steering Council intent to approve PEP 703: ma... by tgv on Saturday July 29, 2023 @03:59AM (#36917961)
I think you're right. Making it an explicit opt-out, as is planned for the first stage, should be fine. Expecting to make it opt-in in 5 years seems too optimistic to me. It relies on all the library developers to fix their libraries (also the Python ones). That's tough work, and importantly, if done well, it will even go unappreciated: nobody will notice it.

Many libraries have never had a multi-processing use case, others are so big that bugs are bound to happen, many of them subtle, so one guaranteed outcome will be unreproducible complaints and devs throwing in the towel. Opt-in will make people unhappy.

Re:The Python Steering Council intent to approve PEP 703: ma... by Arbortheus on Saturday July 29, 2023 @06:34AM (#36918788)
Surely if it goes well people will see their existing python codebases become more performant with no development required aside from updating some dependencies? Not nobody will notice it.

It seems like it could be a great outcome for developers.

Re:The Python Steering Council intent to approve PEP 703: ma... by tgv on Saturday July 29, 2023 @01:01PM (#36922193)
So, removing the GIL will slow down Python, at least initially. Speed gains from multi-threading won't come if you don't change your code, unless you happen to use a library that becomes multi-threaded (in which case you have to start worrying about callbacks).

> It seems like it could be a great outcome for developers.

For the ones that use certain libraries, not for those that have to build them.

Re:The Python Steering Council intent to approve PEP 703: ma... by xmcqdpt2 on Saturday July 29, 2023 @07:06AM (#36919007)
Most C libraries that are often called from python also have bindings for other languages and so at this point should be threadsafe, right? Also even GIL python supports threads so all these libraries will at least be reentrant safe already.

For libraries that are reentrant but not thread safe, it should be sufficient to just add a global lock wrapping every call, which is pretty close to what the GIL was doing anyway.

It seems to me that in many (most?) cases it should be relatively straightforward to make existing libraries work without the GIL (at the cost of parallelism). I guess the main issue will be for libraries that call back into the python runtime from the C side.