It's not specific to Python. If you want to do fine-grained "parallelism" on a set of complex shared datastructures, your cores might look busy, but really you're just wasting most of the time locking/unlocking stuff. That's not necessarily going to result in a speedup over a lock-free single-threaded implementation (that you didn't implement to compare against).

In the case of Python, the single-threaded implementation was already there, the implementation that removed the GIL in favor of fine-grained locking failed to deliver the goods.

If you want to actually take advantage of parallelism, you want data structures that are amenable to it, which is basically sections of arrays without a lot of potential read/write conflicts over multiple threads. If you're at that point, you probably want to use C/C++/Rust anyway, you don't want speed up something that's already 100x slower (the interpreter) by parallelizing it. Python offers that with C-extensions like NumPy.

> your cores might look busy, but really you're just wasting most of the time locking/unlocking stuff.

Not if you're reading most of the time and are using a modern concurrent GC.

It doesn't matter if you're reading or writing, if there's a potential conflict you need to lock, or you need some complex and probably buggy "lock-free" data structure. The complexity and memory overhead of a concurrent GC just goes on top of that.

Locking an uncontended lock is cheap (at least compared to a contended one) so actual conflict does matter more than potential conflict.