You may think you don’t need it but I suspect if you program larger than trivial python applications there comes a point where you do but you don’t even think about how the GIL is restricting you. Think about load time for example of something that does a bunch of work to initialise. Without the GIL it would be relatively trivial to speed up things that are independent, while with GIL you are usually out of luck because you can have either shared memory (e.g. previously loaded state) or concurrency but not both at the same time. From experience, trying to serialise and use multiprocessing is often eating up all the potential gains.

I would love to parallelize a plugin script in Cura, the 3D print slicer. It does a bunch of embarrassingly parallel calculations, and could be made at least 16x faster for me. Because it's a plugin, though, it isn't pickle-able and multiprocessing doesn't work. I managed to make it work in a branch, but only on OSes that can fork processes. On windows, the plugin spawns multiple GUIs because importing the cura package apparently has the side effect of launching it...

If there wasn't the GIL, I could just create a thread pool and be done, and Cura could continue to be a delightful mess. :-)

20-odd years ago, my team solved some performance issues in our desktop app by splitting a couple of tasks into their own threads on a computer with a single CPU. Being able to write threaded code is really useful.

Eh, even with true multithreading python will still be slow as molasses. If you want speed python will still not be your choice.

Some programming tasks are so trivial that processing speed is completely uninteresting.

Some are so compute heavy that massive amounts of time is put into tuning them (sort algorithms, fast-fourier transforms, etc).

Most fall somewhere on the spectrum between those extremes. If you can speed up your program by a factor of 20 by adding threads, Python can cover a bit more of that in-between spectrum.

I'm not sure I fully buy this argument. I think programming languages are chosen for projects based on a number of different factors, speed being one of them, but developer productivity being another (and there are certainly many more). Given that there isn't usually going to be one language that is the best choice for every single factor, it seems pretty natural that some teams might pick Python due to productivity being more important but still have some need for better performance, and other teams might not be able to compromise on performance and have to resort to something like Java or Go or C++ but still would be able to iterate faster with something higher level. It's definitely not a given that there are enough potential projects that would get enough benefit from removing the GIL to make it worth it, but it seems silly to claim that anyone would would get any possible benefit from performance would never choose Python for other reasons.

There are enough choices that offer Python's productivity alongside JIT/AOT options, better supported from the community than PyPy.

As for the C, C++, Fortran libraries with Python bindings, any language with FFI can call into them as well.

I would say, Python's adoption while lacking performance is what is now building pressure to avoid specific Python communities to leave Python and migrate into one of those ecosystems in search of a better mix of productivity/performance, without being forced to use two languages.

Which other languages do you suggest?

Common Lisp, Scheme, Julia, Scala, Clojure, F#, OCaml

F# deserves its own category. You even get access to the .NET ecosystem for free. The developer experience is something else too.

Same applies to Common Lisp, Scheme, Scala and Clojure on the JVM/JavaScript platforms.

And Clojure also has a .NET implementation.

Ooh, a Clojure .NET implementation? This is news to me. From a quick look at the website, it looks really good... but how is it in practice? One of the major advantages of F Sharp's .NET integration is that it's developed by Microsoft, as of course they pretty much created the ecosystem from scratch.

It doesn't get that many public use cases, here's the biggest one I've seen: http://arcadia-unity.github.io/

But it is being continuously kept up to date by Cognitect, along with active development on a clojure-clr-next version. Maybe it has a bigger population of non public users.

Also: here's another implementation of Clojure on CLR, "Morgan And Grand Iron Clojure Compiler": https://github.com/nasser/magic (see also http://nas.sr/magic/) of which there's a great cmpiler implementation talk (at Clojure/North) somewhere on Youtube.

Performance is not binary. If you can be productive with Python and get closer to your performance goal, you get the best of both worlds. A lot of people who aren't primarily programmers don't have time to write everything in C++ (or keep up with development in that area). A more performant Python is a huge win for everybody.

Just like language choice isn't binary, there are plenty of options with Python like productivity, with much better performance, it isn't Python vs C++.

Not with a rich ecosystem like Python's.

Quality matters more than quantity, and in that regard there are enough options available.

This is a tire fire of a thread, it's clear there's lots of confusion about the tradeoffs.

This isn't a case of "x" or "y". There is literally nothing valuable about the GIL, it's an ugly hack of a vestigial appendage. Perhaps the reason I'm familiar with it is because the lack of elegant MP threading in Python perturbed me for years, until I was introduced to Golang.

Python devs generally don't want to use Java, JavaScript, etc. And the Go ecosystem is good but not as rich as Pythons.

Anyhow, take care pjmlp. Until our paths cross again I wish you all the best!

As long as the use cases one cares about are covered, the amount of additional packages are needless noise.

JavaScript, Java, .NET, C and C++ are where to look for, if you want to count package numbers, for AOT/JIT languages.

If python can get a trivial 4x speedup by replacing a for loop with a threadpool.map, then that is worth it to many python programmers.

Just because python isn't the fastest language in the world, does not mean that making it easier / possible to write faster python is worthless.

a few years back I was curious so I took a self-balancing robot that I had written in C (it did PID in real time to set motors in response to the current angle).

I ported it to Python, with a totally naive simple single threaded loop. It worked perfectly. 25 updates a second, forever. No C code, except the interpreter doing its thing, and some GPIO code.

That's not slow as molasses.

From upthread:

> My page load time dropped from 77ms to 47ms when I ran some of the code in parallel.

I suspect some of us welders will also find some improvements here.

That's just a 2x speed increase.

More than 5x can be achieved with the GIL included (and has been promised, by the new Microsoft Python-speed-up initiative and that Python dev who made a similar proposal).

That Python dev (Mark Shannon) is part of the Microsoft team :-)

> I don’t think I’ve ever needed compute-concurrency, not in Python anyway.

I have. Some very inexperienced “senior engineers” at my last gig thought it would be fine to build an analytics platform in Python because “Pandas will make it fast”. Unfortunately even modest datasets would cause timeouts and even seize the event loop, while a naive Go prototype would finish in a few hundred milliseconds.

> Remember when it was discovered that arc welders could produce Carbon 60, and there was a huge run on arc welders?

No, I don't, and can't find any writing about this at all, do you have any links? Because that sounds amazing

Here's the text of a patent describing the process. [0] Googling "fullerene" will get you more useful hits than C-60, so, e.g. "generating fullerene from arc welding."

[0] https://patents.google.com/patent/US5227038A/en

Bing found two URLs... Both to this page. Google found one -- also here -- and two ads.

This:

   generate fullerene "arc weld"

Worked a bit better in both. N.B: Quote marks only around the "arc weld" bit.

I believe the original paper on Buckminsterfullerine aka Bucky balls aka C60 described the approach. Basically do violent stuff with carbon and stick it in a mass spectrometer. The violent carbon bit is arc welding, because the arc welding tips are made of graphite.

Wikipedia informs me that the first papers used a laser (https://www.nature.com/articles/318162a0 - "graphite has been vaporized by laser irradiation, producing a remarkably stable cluster consisting of 60 carbon atoms") (1985) and the arc welding approach was a few year later. The paper "Solid C60: a new form of carbon" (1990) paper describes using benzene to dissolve the soot to extract the C60 (https://heyokatc.com/pdfs/MISC/Solid_C60-_a_new_form_of_carb...).