Indeed, but Python is used to orchestrate all these lower-level libraries. If you have Python on top, you often want to call these libraries on a loop, or more often, on parallelized multi-stage pipelines.

Overhead and parallelization limitations become a serious issue then. Frameworks like PySpark take your Python code and are able to distribute it better, but it's still (relatively) very slow and clunky. Or they can limit what you can do to a DSL (often SQL), but you can't to much serious data work without UDFs, where again Python comes in. There are tricks but you can never really avoid the limitations of the Python interpreter.

Most of these native libraries' output isn't 1-1 mappable to Python. Based on the data you need to write native data wrappers, or worse, marshal the data into managed memory. The overhead can be high.

It gets worse because Python doesn't expose you to memory management. This initially is an advantage, but later on causes bloat.

Python is an incredibly easy interface over these native libraries, but has a lot of runtime costs.

Yet another reason to use native compiled languages with bindings to the same C and C++ libraries.

If using C++20 onwards, then it is relatively easy to have similar high level abstractions, one just needs to let go of Cisms that many insist in using.

Here Rust has clearly an advantage that it doesn't allow for copy-paste of C like code.

Naturally D and Swift with their safety and C++ interop, would be an option as well.

Have you had any success using LLMs to rewrite Python to rust?

They’re very good at porting code between languages but going from a dynamically typed language with a large standard library to a static one with a large library ecosystem requires a bit more hand holding. It helps to specify the rust libraries you want to use (and their versions) and you’ll probably want to give a few rounds of feedback and error correction before the code is ready.

> Python is an incredibly easy interface over these native libraries, but has a lot of runtime costs.

It also means that many people use Python while they don't understand that what part of the code is actually fast. They mix Python code with wrappers to native libraries, and sometimes the Python code slows down the overall work substantially and people don't know that fault is there. E.g use Python Maths with the mix of Numpy math bindings, while they can do it with Numpy alone.