[1]: https://www.youtube.com/watch?v=-DVyjdw4t9I

[2]: https://news.ycombinator.com/item?id=36643670

That doesn't follow; you're assuming that Python should prioritize performance as a top priority, which is very much not a given. Python has always excelled at being easy to use, being flexible, being a great glue language - but performant? An interpreted, dynamically typed language? That's like making a C interpreter - you can do it, but that doesn't make it a good idea.

And yet despite your complaints, JavaScript, Lua and Julia continue to rise in popularity.

https://discuss.python.org/t/a-fast-free-threading-python/27...

A lot will depend on the Python community and the distribution channels. I could see the community struggling to adopt it in a timely fashion, or distributions jumping the gun (Ubuntu, Fedora, Anaconda). Maybe it's too early to make hard decisions, but how much control does the SC really have to avoid such a scenario?

But apparently Python needs some "success" and this makes a good bullet point. Correctness does not really matter in the Python world.

The unicode/text changes alone were a pretty good reason. Division producing floats are also a nice change IMO. I don’t want to discount the challenges with the transition but saying there was no good reason isn’t right to me.

That is too long for Python to have 2 modes. Half a decade is more than enough time for 2 modes to become the status quo. For one thing, think of all the outdated Stackoverflow threads that will be hanging around after that 5 years. I am not optimistic that 5 years won't turn into 10 years of uncertainty and breakage.

But 5 years might be too short for everyone to dredge up all that C code, update it, test it, and call it mature.

I guess we'll see.

Developers are lazy. Most people will just do nothing until, 5 years from now, some blog post will go "oh, next month we switch noGil as default, good luck!". At that point everyone will scramble, rush out buggy releases, and spend 5 years finding all the problems.

I'm afraid we'll soon learn its not the version # that's burdensome, but the real or perceived(!) incompatibility between versions.

I wonder if introducing such a monumental change in a build flag of the minor version is really wise. Certainly its not in line with any interpretation of semantic versioning (to be fair I think the PSF does not claim to use that).

PYTHONGIL is an awkward tri-state. 0, 1, and unset all do different things. Wouldn’t some self-explanatory strings be better? PYTHONGIL=auto for the default, force-gil and force-nogil for the forced modes.

What is the use case for this?

Who needs it?

I _am_ a python programmer.

> What is the use case for this?

> Who needs it?

This can be answered with a simple user story:

As "THE PYTHON STEERING COUNCIL",

I want to "GET RID OF THE GIL",

so that "PEOPLE WILL STOP WHINGING ABOUT THE GIL."

To be fair, there's another group of people who stand to benefit. Namely, any python programmers who currently believe that "threads are an easy way to make my program go faster" will soon be the recipients of a valuable learning experience.

;)

- library author marks their library "no-GIL" after making sure it's thread-safe without GIL

- if the interpreter sees this metainformation, it temporarily disables GIL for the current OS thread while running the library's code

- result: old versions of Python can still run no-GIL libraries under GIL, while new versions of Python allow to gradually remove GIL

Or it's not how CPython works?

They will approve something if it serves a corporation. The submission here is likely CYA, so they can say that "they asked the community".

There is no appreciation for people doing grassroots open source software. If Instagram can add another hack instead of switching to Java, it will be approved.

It is important to remember that paid corporate developers will have job security every time new pain is introduced in the Python ecosystem.

Some of those do sufficiently simple things that they can run without any locking and all will be fine.

Others will still need locking, but are now under pressure to run without the gil. Some of those are going to do DIY locking within their own bounds.

Maybe what python has really been missing all these years is loads of ad hoc mutex calls scattered across the ecosystem. Data races and deadlocks introduced in the name of performance is not how I expected python to go out.

edit: expanding on this pessimism a bit.

Making C libraries written assuming a global lock thread safe is the sort of thing I'd expect concurrency experts to advise against and then make mistakes while implementing it. My working theory is that most people who wrote C extensions for python are not concurrency experts and are great programmers who won't back down from a challenge.

The data-race/hang/segfault consequences of this combination look totally inevitable to me. Python application developers are not going to love the new experience and I'm thankful my products are not built on top of the python ecosystem.

Many libraries have never had a multi-processing use case, others are so big that bugs are bound to happen, many of them subtle, so one guaranteed outcome will be unreproducible complaints and devs throwing in the towel. Opt-in will make people unhappy.

It seems like it could be a great outcome for developers.

> It seems like it could be a great outcome for developers.

For the ones that use certain libraries, not for those that have to build them.

For libraries that are reentrant but not thread safe, it should be sufficient to just add a global lock wrapping every call, which is pretty close to what the GIL was doing anyway.

It seems to me that in many (most?) cases it should be relatively straightforward to make existing libraries work without the GIL (at the cost of parallelism). I guess the main issue will be for libraries that call back into the python runtime from the C side.

Ruby could learn a lot once this is done, but at the moment GIL optional in python seems to be a 2030 goal.

I greatly welcome true multithreading in Python.

I use both multiprocessing and multithreading in Python for different projects. See [0] for my multiprocessing example and python Threads for IO heavy tasks in [1]. But it would be far more efficient to use true threads.

Threads can communicate any amount of data in a single atomic almost instant operation. Using the local loopback interface or multiprocessing or pipes, this is not possible.

I am working on a multithreading architecture I call three tier multithreading architecture

https://github.com/samsquire/three-tier-multithreaded-archit...

My goal is extremely scalable and performant servers but Python is probably the wrong job for that.

[0]: https://news.ycombinator.com/item?id=36897054 (my description of my use of multiprocessing) [1]: https://devops-pipeline.com/ (my use of multithreading)

1. There are some improvements worth breaking reverse-compatibility for, and removing the GIL is such an improvement. Whether the changes in Python 3 were worth making breaking changes for is debatable: certainly I don't see "print" being a function as particularly valuable. But the flipside is that the 2-to-3 transition was overblown by a vocal minority. I've transitioned more than 5 codebases from 2 to 3, and in most cases, there were few problems. Most problems were with codebases where previous developers had pulled in libraries for everything, resulting in an amalgamation of abandoned libraries, but these codebases run into problems even without the core language breaking compatibility. The answer isn't to flame your language into never breaking compatibility, it's to not import all of pip and expect that to be a sustainable strategy.

The situation we have now is that the steering committee has received so much heat from the vocal minority that they're terrified to make breaking changes. But removing the GIL should be a breaking change. It's too fundamental to how Python works to not be. So they're trying to remove the GIL and make it not a breaking change, which is a bad idea, because it is ultimately going to be a breaking change. It would be much better to admit this is a breaking change and start working on the transition plan, than to try the impossible task of making it not breaking because you're too terrified of your users to admit the truth.

We've already seen this in Python 3.11 which broke code in my codebase. The changes to fix the breakage weren't hard, but I would have liked better communication that this might happen. But I also understand why this was hidden in a deprecation warning in a minor release rather than publicized, because the Python team is probably tired of being flamed for making breaking changes.

2. The more fundamental problem here is that a lot of other features of Python were built around the GIL. Most obviously, the async paradigms makes sense largely because of the GIL. Sans-GIL, it looks like in retrospect a send/recv actor model a la Erlang would have been a much better way forward. It's not really possible to reverse this, and this might be pushing Python toward a less cohesive set of features that don't really make sense together. This makes it feel like this is too little too late.

If this doesn't ring any alarm bells I don't know what will.

> Basically for the moment the GIL-less idea would likely be burdensome for us and the fact that it's only an "option" seems to strongly imply major compatibility issues that we would not prefer.

https://github.com/sqlalchemy/sqlalchemy/discussions/10002#d...