Very limited view which ignores way too many areas.
Really, for almost any X, somebody somewhere has to do X with strict performance requirements (or at very large scale, so better perf == savings)
Most of these python libraries are only fast for relatively large and relatively standard operations in the first place. If you have a lot of small/weird computations, they come with a ton of overhead. I've personally had to write my own fast linear algebra libraries since our hot loop was a sort of modified tropical algebra once.
Rather unfortunately, my current bugbear is that Pytorch is... slow. On the CPU. One of the most common suggestions for people who want stable diffusion to be faster is, wait for it, "Try getting a recent Intel CPU, you'll see a real uplift in performance".
This despite the system only keeping a single CPU core busy. Of course, that's all you can do in Python most of the time.
(You can also use larger batch sizes. But that only partially papers over the issue, and also it uses more GPU memory.)
Also if I (a programmer) want to write really really fast code I'm probably reaching for tools like tensorflow, numpy, or jax. So there's not much incentive for me to switch to a more efficient language when as near as I can tell the best tooling for dealing with SIMD or weird gpu bullshit seems to be being created for python developers. If you want to write fast code do it in c/rust/whatever, if you want to write really fast code do it in python with <some-ML-library>.
For a very specific definition of the word "fast" at least.