When you have so many large companies with a vested interest in optimization I believe that Python can become faster by doing realistic and targeted optimizations . The other strategies to optimize didn’t work at all or just served internal problems at large companies .
What exactly do you do with Python that slows you down ?
I still use Python for cases when dev time is more important than execution time (which is rarer now that I'm working adjacent to "big data") or when I'm doing things like writing python to close gaps in the various arrays of web apps provided for navigating the corporate work flow, and if I went as fast as a 64 core box let me, we'd have some outages in corp github or artifactory or the like, so I just do it one slow thing at a time on 1 core and wait for the results. Maybe multiprocessing with 10 process worker pool once I'm somewhat confident in the back end system I am talking to.
(edit: removed my normal email signoff)
Don't take this the wrong way but I think you could be more specific. Are you saying that similar to Go it should be just faster in general?
So many successful projects/technologies started out that way. The Web, JavaScript, e-mail. DNS started out as a HOSTS.TXT file that people copied around. Linus Torvalds announced Linux as "just a hobby, won't be big and professional like gnu". Minecraft rendered huge worlds with unoptimized Java and fixed-function OpenGL.
I took a graduate level data structures class and the professor used Python among other things "because it's about 80 times slower than C, so you have to think hard about your algorithms". At scale, that matters.
It prevents you from taking advantage of multiple cores. Doesn't really impact straight-line execution speed.
A data structures course is primarily not going to be concerned with multithreading.
Most computers running Python today will have multiple cores. If your program can only use a fraction of its available computing power, it affects the program's execution speed. Python is too widely used in compute-intensive applications (e.g. machine learning) for GIL-related issues to be ignored in the long term.
Ideally, Python should have automatic load scaling and thread-safe data structures in the standard library to take advantage of all available CPU cores.
Java has had concurrent data structures in its standard library for years and is adding support for "easy" multi-threaded execution with Project Loom.
Python needs to add its own Pythonic versions of thread-safe data structures and compute load scaling to take advantage of the multi-core CPUs that it runs on.
>> A data structures course is primarily not going to be concerned with multithreading.
A graduate-level data structures course might include concurrent and parallel data structures.
Being fast isn't contradictory with this goal. If anything, this is a lesson that so many developers forget. Things should be fast by default.
I think some people can do this and some can't. For some, writing slow code is much easier, and their contributions are still valuable. Once the bottlenecks are problems, someone with more performance-oriented skills can help speed up the critical path, and slow code outside of the critical path is just fine to leave as-is.
If you somehow limited contributions only to those who write fast code, I think you'd be leaving way too much on the table.
There are a few much better options like mergesort or quicksort, but they have their tricks.
But to sort real data really fast, you should use something like timsort, that detects if the data is just the union of two (or a few) sorted parts, so it's faster in many cases where the usual sorting methods don't detect the sorted initial parts. https://en.wikipedia.org/wiki/Timsort
Are you sorting integers? Strings? Ascii-only strings? Perhaps the code should detect some of them and run an specialized version.
It absolutely is contradictory. If you look at the development of programming languages interpreters/VMs, after a certain point, improvements in speed become a matter of more complex algorithms and data structures.
Check out garbage collectors - it's true that Golang keeps a simple one, but other languages progressively increase its sophistication - think about Java or Ruby.
Or JITs, for example, which are the latest and greatest in terms of programming languages optimization; they are complicated beasts.
Go’s GC was intentionally slow at first. They wanted to get it right THEN make it fast.
No offense but you’re not making a strong case. You’re sounding like an inexperienced coder that hasn’t yet learned that premature optimization is bad.
Of course one can't release optimal code from version one, that would be absurd.
Also your last sentence is extremely condescending.
In over 90% of my work in the SW industry, being fast(er) was of no benefit to anyone.
So no, it should not be fast by default.
Maybe better to elaborate on what it should be, if not fast? Surely you aren’t advocating things should be intentionally slow by default, or carelessly inefficient?
There’s a valid tradeoff between perf and developer time, and it’s fair to want to prioritize developer time. There’s a valid reason to not care about fast if the process is fast enough that a human doesn’t notice.
That said, depends on what your work is, but defaulting to writing faster, more efficient code might benefit a lot of people indirectly. Lower power is valuable for server code and for electricity bills and at some level for air quality in places where power isn’t renewable. Faster benefits parallel processing, it leaves more room for other processes than yours. Faster means companies and users can buy cheaper hardware.
It should satisfy the needs of the customer and it should be reasonably secure.
Everything else is a luxury.
My point was that in most of my time in the industry being faster would not have benefited my customers in any manner worth measuring.
I'm not anti-performance. I fantasize about how to make my software faster to maintain my geek credentials. But neither my bosses nor my customers pay for it.
If a customer says they want it faster we'll oblige.
Languages that tried to be all things to all people really havent done so well.
-- Donald Knuth
https://softwareengineering.stackexchange.com/a/80092
> Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.
I'm confused. What do you think the context changes? At least as I read it, both the short form and full context convey the same idea.
Instead, what he meant is that you should profile what part of the code is slow and focus on it first. Knuth didn't say you should be fine with 10-1000x slower code overall.
Yes, if you quote Knuth here (whether the short quote or a longer version) you are probably responding to someone whom you believe is engaged in premature optimization.
It remains that the person quoting Knuth isn't claiming that there isn't such a thing as justified optimization. As such, pointing to the context doesn't really add to the conversation. (Nor does a thoughtless quote of Knuth either)
But either way I think the original statement is so uncontroversial and common-sense I actually think it doesn't help any argument unless you're talking to an absolutely clueless dunce or unless you're dealing with someone who somehow believes every optimization is premature.
As was said, profile, find user-impacting bottlenecks, and then optimize.
But, they aren't.
I mean, profile, identify critical components, and rewrite in C (or some other low-level language) for performance is a normal thing for most scripting languages.
> You've got to decide upfront whether the trade off in a rapid prototyping language is good or not, not wait until you've written the project and then profile it.
No, you absolutely don't.
Or to look at it another way: Python with C extensions is effectively another language. You have to consider it as an option along with Pure Python, Rust, Go, C++, Java, FORTRAN, or what have you. Each language has different trade-offs in development time vs performance.
a) gives a rule-of-thumb estimate of how much optimisation to do (maybe 3% of all opportunities);
b) explains that non-premature opimisation is not just not the root of all evil but actually a good thing to do; and
c) gives some information about how to do non-premature optimisation, by carefully identifying performance bottlenecks after the unoptimised code has been written.
I agree with GP that unless we know what Knuth meant by "premature" it is tempting to use this quote to justify too little optimisation.
* Some of the many applications relying on these pieces, could almost certainly use the speedup and for those it wouldn't be premature
* The return of investment is massive due to the scale
* There are tremendous productivity gains by increasing the performance baseline because that reduces the time people have to spend optimizing applications
And it's got commercial interests willing to throw money at performance now because of that.
Seems like the Python community, whether as top-down strategy or emergent aggregate of grassroots decisions made the right choices here.
Languages don't succeed on their technical merit. They succeed by being good enough to gain traction, after which it is more about market forces. People choose Python for it's great ecosystem and the availability of developers, and they accept the price they pay in performance. But that doesn't imply that performance wasn't an issue in the past, or that Python couldn't have been even more successful if it had been more performant.
And to be clear, I use Python every day, and I deeply appreciate the work that's been put into 3.10 and 3.11, as well as the decades prior. I'm not interested in prosecuting the decisions about priorities that were made in the past. But I do think there are lessons to be learned there.
In my experience it's far more common for "optimizations" to be technical debt than the absence of them.
> only ever think about performance in retrospect
From the extra context it pretty much does mean that. "but only after that code has been identified" - 99.999% of programmers who think they can identify performance bottlenecks other than in retrospect are wrong, IME.
It is not possible to predict bottlenecks in computation, no. But the implications of putting global state behind a mutex in a concurrent application should be clear to the programmer, and they should think seriously before making a choice like that. If you think of a different way to do that while the code is still being written, you'll avoid trapping yourself in an irreversible decision.
Note that all of the given benchmarks are microbenchmarks; the gains in 3.11 are _much_ less pronounced on larger systems like web frameworks.
Because the core team just hasn't prioritized performance, and have actively resisted performance work, at least until now. The big reason has been about maintainership cost of such work, but often times plenty of VM engineers show up to assist the core team and they have always been pushed away.
> Now let's get a sane concurrency story
You really can't easily add a threading model like that and make everything go faster. The hype of "GIL-removal" branches is that you can take your existing threading.Thread Python code, and run it on a GIL-less Python, and you'll instantly get a 5x speedup. In practice, that's not going to happen, you're going to have to modify your code substantially to support that level of work.
The difficulty with Python's concurrency is that the language doesn't have a cohesive threading model, and many programs are simply held alive and working by the GIL.
From the very page you've linked to:
"Faster CPython explores optimizations for CPython. The main team is funded by Microsoft to work on this full-time. Pablo Galindo Salgado is also funded by Bloomberg LP to work on the project part-time."
This is in very active development[1]! And seems like the Core Team is not totally against the idea[2].
[1] https://github.com/colesbury/nogil
[2] https://pyfound.blogspot.com/2022/05/the-2022-python-languag...
* Statically allocated ("frozen") core modules for fast imports
* Avoid memory allocation for frames / faster frame creation
* Inlined python functions are called in pure python without needing to jump through C
* Optimizations that take advantage of speculative typing (Reminds me of Javascript JIT compilers -- though according to the FAQ Python isn't JIT yet)
* Smaller memory usage for frames, objects, and exceptions
Dang that certainly does sound like low hanging fruit. There's probably a lot more opportunities left if they want Python to go even faster.
Given that Python is interpreted, it's quite unclear what this could mean.
Also, what does it mean to "call" an inlined function?? Isn't the point of inline functions that they don't get called at all?
> During a Python function call, Python will call an evaluating C function to interpret that function’s code. This effectively limits pure Python recursion to what’s safe for the C stack.
> In 3.11, when CPython detects Python code calling another Python function, it sets up a new frame, and “jumps” to the new code inside the new frame. This avoids calling the C interpreting function altogether.
> Most Python function calls now consume no C stack space. This speeds up most of such calls. In simple recursive functions like fibonacci or factorial, a 1.7x speedup was observed. This also means recursive functions can recurse significantly deeper (if the user increases the recursion limit). We measured a 1-3% improvement in pyperformance.
switch (op) {
case "call_function":
interpret(op.function)
...
}
switch (op) {
case "call_function":
... setup frame objects etc ...
pc = op.function
continue
....
}
Python is compiled. CPython runs bytecode.
(If Python is interpreted, then so is Java without the JIT).
Because it's developed and maintained by volunteers, and there aren't enough folks who want to spend their volunteer time messing around with assembly language. Nor are there enough volunteers that it's practical to require very advanced knowledge of programming language design theory and compiler design theory as a prerequisite for contributing. People will do that stuff if they're being paid 500k a year, like the folks who work on v8 for Google, but there aren't enough people interested in doing it for free to guarantee that CPpython will be maintained in the future if it goes too far down that path.
Don't get me wrong, the fact that Python is a community-led language that's stewarded by a non-profit foundation is imho its single greatest asset. But that also comes with some tradeoffs.
Who is working on python voluntarily? I would assume that, like the Linux kernel, the main contributors are highly paid. Certainly, having worked at Dropbox, I can attest to at least some of them being highly paid.
second, CPython is just a C interpeter, there isn't much assembly if any.
third, contributing to CPython is sufficiently high-profile you could easily land a 500k-job just by putting it on your CV.
No, those are not the real reasons why it hasn't happened before.
I think the word you're looking for is "politics".
The underlying source of the Politics of Python and associated perceptions stems from the core team's culture of not being "friendly".
They have no obligation to go out of their way to cater to yours.
The entitlement of the new generation of open-source contributors to require political correctness or friendliness is so destructive it's ridiculous. I wouldn't want to be involved with any project that prioritizes such zealotry on top of practicality.
The world constantly changes, projects adapt or become less useful.
The developer who worked at Microsoft to make iron python I think that was his full-time project as well and it was definitely faster than cpython at the time
Not because they deliberately only wanted to build an alternative interpreter.
But if it was just core-team resistance and not a more fundamentally different objective than just improving the core interpreter performance, maintaining support for native extensions while speeding up the implementation would have been a goal even if it had to be in a fork.
They were solving a fundamentally different (and less valuable to the community) problem than the new faster CPython project.
That might very well be, but I talked about UnladdedSwallow and such "native" attempts.
Not attempts to port to a different ecosystem like Jython or IronPython or the js port.
>maintaining support for native extensions while speeding up the implementation would have been a goal even if it had to be in a fork.
That's not a necessary goal. One could very well want to improve CPython, the core, even if it meant breaking native extensions. Especially if doing so meant even more capabilities for optimization.
I, for one, would be fine with that, and am pretty sure all important native extensions would have adapted quite soon - like they adapted to Python 3 or the ARM M1 Macs.
In fact, one part of the current proposed improvements includes some (albeit trivial) fixes to native extensions.
Really the value came from the thread safe container/collections data types which are the foundation of high performance programming
Current changes don't really break it.
Given the popularity, you’d think Python would have had several generations of JITs by now and yet it still runs interpreted, AFAIK.
JavaScript has proven that any language can be made fast given enough money and brains, no matter how dynamic.
Maybe Python's C escape hatch is so good that it’s not worth the trouble. It’s still puzzling to me though.
Yeah but commercial Smalltalk proved that a long time before JS did. (Heck, back when it was maintained, the fastest Ruby implementation was built on top of a commercial Smalltalk system, which makes sense given they have a reasonably similar model.)
The hard part is that “enough money“ is...not a given, especially for noncommercial projects. JS got it because Google decided JavaScript speed was integral to it's business of getting the web to replace local apps. Microsoft recently developed sufficient interest in Python to throw some money at it.
Google wasn’t alone in optimizing JS, it actually came late, Safari and Firefox were already competing and improving their runtime speeds, though V8 did doubled down on the bet of a fast JS machine.
The question is why there isn’t enough money, given that there obviously is a lot of interest from big players.
I'd argue that there wasn't actually much interest until recently, and that's because it is only recently that interest in the CPython ecosystem has intersected with interest in speed that has money behind it, because of the sudden broad relevance to commercial business of the Python scientific stack for data science.
Both Unladen Swallow and IronPython were driven by interest in Python as a scripting language in contexts detached from, or at least not necessarily attached to, the existing CPython ecosystem.
A better question would be to ask where Python is slow today and if it matters to big business, then.
It rules in AI, for instance. Is it still mainly glue code for GPU execution and performance isn’t at all critical there?
Even if it wasn't good, the presence of it reduces the necessity of optimizing the runtime. You couldn't call C code in the browser at all until WASM; the only way to make JS faster was to improve the runtime.
> JavaScript has proven that any language can be made fast given enough money and brains, no matter how dynamic.
JavaScript also lacks parallelism. Python has to contend with how dynamic the language is, as well as that dynamism happening in another thread.
There are some Python variants that have a JIT, though they aren't 100% compatible. Pypy has a JIT, and iirc, IronPython and JPython use the JIT from their runtimes (.Net and Java, respectively).
PyPy did implement a JIT for Python, and it worked really well, until you tried to use C extensions.
But still, it’s kind of surprising that PHP has a JIT and Python doesn’t (official implementation I mean, not PyPy).
Think about it: if you have some Python application that's having performance issues you can either dig into a foreign codebase to see if you can find something to optimize (with no guarantee of result) and if you do get something done you'll have to get the patch upstream. And all that "only" for a 25% speedup.
Or you could rewrite your application in part or in full in Go, Rust, C++ or some other faster language to get a (probably) vastly bigger speedup without having to deal with third parties.
Programmers love to save an hour in the library by spending a week in the lab
Or you can just throw more hardware at it, or use existing native libraries like NumPy. I don't think there are a ton of real-world use cases where Python's mediocre performance is a genuine deal-breaker.
If Python is even on the table, it's probably good enough.
Instead, there are big companies, who are running let's say the majority of their workloads in python. It's working well, it doesn't need to be very performant, but together all of the workloads are representing a considerable portion of your compute spend.
At a certain scale it makes sense to employ experts who can for example optimize Python itself, or the Linux kernel, or your DBMS. Not because you need the performance improvement for any specific workload, but to shave off 2% of your total compute spend.
This isn't applicable to small or medium companies usually, but it can work out for bigger ones.
edit: ray https://github.com/ray-project/ray is also pretty easy to use and powerful for actual parallelism
Do you compare it to threads and pools, or judge it on its merits as an async framework (with you having experience of those that you think are done better elsewhere, e.g. in Javascript, C#, etc)?
Because both things you mention "demands on how you build your code" and "limited scope" are part of the course with async in most languages that aren't async-first.
I don't see how "asyncio is annoying and can only be used for a fraction of scenarios everywhere else too, not just here" is anything other than reinforcement of what I said. OS threads and processes already exist, can already be applied universally for everything, and the pool executors can work with existing serial code without needing the underlying code to contort itself in very fundamental ways.
Python's version of asyncio being no worse than someone else's version of asyncio does not sound like a strong case for using Python's asyncio vs fixing the better-in-basically-every-way concurrent futures interface that already existed.
Well, I didn't try to refute what you wrote (for one, it's clearly a personal, subjective opinion).
I asked what I've asked merely to clarify whether your issue is with Python's asyncio (e.g. Python got it wrong) or with the tradeoffs inherent in async io APIs in general (regardless of Python).
And it seems that it's the latter. I, for one, am fine with async APIs in JS, which have the same "problems" as the one you've mentioned for Python's, so don't share the sentiment.
Both, but the latter part is contextual.
> I, for one, am fine with async APIs in JS
Correct me if you think I'm wrong, but JS in its native environment (the browser) never had access to the OS thread and process scheduler, so the concept of what could be done was limited from the start. If all you're allowed to have is a hammer, it's possible to make a fine hammer.
But
1. Python has never had that constraint
2. Python's asyncio in particular is a shitty hammer that only works on special asyncio-branded nails
and 3. Python already had a better futures interface for what asyncio provides and more before asyncio was added.
The combination of all three of those is just kinda galling in a way that it isn't for JS because the contextual landscape is different.
Which is neither here, nor there. Python had another big constraint, the GIL. So threads there couldn't go so far as async would. But even environments with threads (C#, Rust) also got big into async in the same style.
>2. Python's asyncio in particular is a shitty hammer that only works on special asyncio-branded nails
Well, that's also the case with C#, JS, and others with similar async style (aka "colored functions"). And that's not exactly a problem, as much as a design constraint.
JS never had an option as, as far as I understand, callback based async was already the norm, so async functions were an improvement over what came before. C# wants to be an high performance language, so using async to avoid allocating a full call stack per task is understandable. In python the bottleneck would be elsewhere, so scaling would be in no way limited by the amount of stack space you can allocate, so adding async is really hard to justify.
Obviously the fact that the GIL prevents effient use of threads, so asyncio becomes the way to get more load from a single CPU by taking advantage of the otherwise blocking time.
You are confusing concurrency and parallelism.
> Threading is "implicit" context switching all in the same process/thread
No, threading is separate native threads but with a lock that prevents execution of Python code in separate threads simultaneously (native code in separate threads, with at most on running Python, can still work.)
Concurrent programs have multiple right answers for "Which line of computation can make progress?" Sequential execution picks one step from one of them and runs it, then another, and so on, until everything is done. Whichever step is chosen from whichever computation, it's one step per moment in time; concurrency is only the ability to choose. Parallel execution of concurrent code picks steps from two or more computations and runs them at once.
Because of the GIL, Python on CPython has concurrency but limited parallelism.
It does allow threads to execute concurrently. It doesn't allow them to execute in parallel if they all are running Python code (if at least one is rubbing native code and has released the GIL, then those plus one that has not can run in parallel.)
But when you try to do things that aren't a map-reduce or Pool.map() pattern, it suddenly becomes pretty warty. E.g. scheduling work out to a processpool executor is ugly under the hood and IMO ugly syntactically as well.
Are you talking about this example? https://docs.python.org/3/library/asyncio-eventloop.html#asy...
However, batteries are not included. For example, it provides no HTTP client/server. It doesn't interop with any synchronous IO tools in the standard library either, making asyncio a very insular environment.
For the majority of problems, Go or Node.js may be better options. They have much more mature environments for managing asynchrony.
[1] https://docs.python.org/3/library/asyncio-eventloop.html#asy...
I believe the reason is that python does not need any low-hanging fruits to have people use it, which is why they're a priority for so many other projects out there. Low-hanging fruits attract people who can't reach higher than that.
When talking about low-hanging fruits, it's important to consider who they're for. The intended target audience. It's important to ask ones self who grabs for low-hanging fruits and why they need to be prioritized.
And with that in mind, I think the answer is actually obvious: Python never required the speed, because it's just so good.
The language is so popular, people search for and find ways around its limitations, which most likely actually even increases its popularity, because it gives people a lot of space to tinker in.
Do we have completely different definitions of low-hanging fruit?
Python not "requiring" speed is a fair enough point if you want to argue against large complex performance-focused initiatives that consume too much of the team's time, but the whole point of calling something "low-hanging fruit" is precisely that they're easy wins — get the performance without a large effort commitment. Unless those easy wins hinder the language's core goals, there's no reason to portray it as good to actively avoid chasing those wins.
Oh, that's not how I interpret low-hanging fruits. From my perspective a "low-hanging fruit" is like cheap pops in wrestling. Things you say of which you know that it will cause a positive reaction, like saying the name of the town you're in.
As far as I know, the low-hanging fruit isn't named like that because of the fruit, but because of those who reach for it.
My reason for this is the fact that the low-hanging fruit is "being used" specifically because there's lots of people who can reach it. The video gaming industry as a whole, but specifically the mobile space, pretty much serves as perfect evidence of that.
Edit:
It's done for a certain target audience, because it increases exposure and interest. In a way, one might even argue that the target audience itself is a low-hanging fruit, because the creators of the product didn't care much about quality and instead went for that which simply impresses.
I don't think python would have gotten anywhere if they had aimed for that kind of low-hanging fruit.
What I'm describing, which is the sense I've always seen that expression used as in engineering, and what GP was describing, is: this is an easy low-risk project that have a good chance of producing results.
E.g. If you tell me that your CRUD application suffers from slow reads, the low-hanging fruit is stuff like making sure your queries are hitting appropriate indices instead of doing full table scans, or checking that you're pooling connections instead of creating/dropping connections for every individual query. Those are easy problems to check for and act on that don't require you to try to grab the fruit hard-to-reach fruit at the top of the tree like completely redesigning or DB schema or moving to a new DB engine altogether.
Easily obtained gains; what can be obtained by readily available means
https://www.merriam-webster.com/dictionary/low-hanging%20fru...
the obvious or easy things that can be most readily done or dealt with in achieving success or making progress toward an objective
"Maria and Victor have about three months' living expenses set aside. That's actually pretty good …. But I urged them to do better …. Looking at their monthly expenses, we found a few pieces of low-hanging fruit: Two hundred dollars a month on clothes? I don't think so. Another $155 for hair and manicures? Denied."
"As the writers and producers sat down in spring 2007 to draw the outlines of Season 7, they knew, Mr. Gordon said, that most of the low-hanging fruit in the action genre had already been picked."
"When business types talk about picking low-hanging fruit, they don't mean, heaven forbid, doing actual physical labor. They mean finding easy solutions."
https://dictionary.cambridge.org/dictionary/english/low-hang...
something that is easy to obtain, achieve, or take advantage of
"The easy changes have all been made. All the low-hanging fruit has been picked."
"When cutting costs, many companies start with the low-hanging fruit: their ad budgets."
"For the beauty-care industry, the teen demographic is a new category for them - low-hanging fruit."
"I'm a great believer in picking low-hanging fruit. Start with what's easy, and go higher later."
"This legislation is some of the low-hanging fruit - the issues that we can agree upon across parties and regions."
If everyone knows it's never going to reach the performance needed for high performance work, and there's already an excellent escape hatch in the form of C extensions, then why would people be spending time on the middle ground of performance? It'll still be too slow to do the things required, so people will still be going out to C for them.
Personally though, I'm glad for any performance increases. Python runs in so much critical infrastructure, that even a few percent would likely be a considerable energy savings when spread out over all users. Of course that assumes people upgrade their versions...but the community tends to be slow to do so in my experience.
Ah! So they are so tall that picking the low-hanging fruit would be too inconvenient for them.
Talk about stretching an analogy too far.
Yes, and I would argue it already exists and is called Rust :)
Semi-jokes aside, this is difficult and is not just about removing the GIL and enabling multithreading ; we would need to get better memory and garbage collection controls. Parts of what's make python slow and dangerous in concurrent settings are the ballooning memory allocations on large and fragmented workloads. A -Xmx would help.
In addition to the other point here about speed not being a target in the first place.
And why they were not addressed: because starting a certain points, the optimization are e.g. processor specific or non intuitive to understand. Making it hard to maintain vs a simple straight forward solution.
Having read about some of the changes [1], it seems like the python core committers preferred clean over fast implementations and have deviated from this mantra with 3.11.
Now let's get a sane concurrency story (no multiprocessing / queue / pickle hacks) and suddenly it's a completely different language!
[1] Here are the python docs on what precisely gave the speedups: https://docs.python.org/3.11/whatsnew/3.11.html#faster-cpyth...
[edit] A bit of explanation what I meant by low-hanging fruit: One of the changes is "Subscripting container types such as list, tuple and dict directly index the underlying data structures." Surely that seems like a straight-forward idea in retrospect. In fact, many python (/c) libraries try to do zero-copy work with data structures already, such as numpy.