Python here basically works as a binder for C, which is what everything is written in.

That is in fact the problem, yes.

Honestly I think julia missed the boat. There was an opportunity for julia to treat the gpu as an abstract distributed node (which is a first class concept in julia) and pay respect to the reality of data gravity. But they chose to instead basically treat the gpu as a synchronous entity.

I'm now bullish on elixir-nx because I think there's an outside shot they will get it right.

Python is a lost cause.

I'm not sure that's necessarily the domain of a low-level package like CUDA.jl though (which I assume you're referring to). That kind of interface is more the domain of higher-level packages like https://github.com/JuliaGPU/DaggerGPU.jl and to a lesser extent https://juliagpu.github.io/KernelAbstractions.jl/stable/. Moreover, the jury is still out on whether the built-in Distributed module is an ideal abstraction for every use-case (clusters, heterogeneous compute, etc.)

WRT Nx, my biggest question is how they'll crack the problem of still needing big balls of C++ and the shims everywhere to get acceleration. Creating a compiler that generates efficient GPU or other accelerator code is a massive research project with no clear winners, never mind the challenge of reconciling the very mutation-heavy needs of GPU compute with a mostly immutable language model.

There's plenty of mutability escape hatches in the erlang vm, and basically everyone who works in the language long enough is familiar with the idea of holding on to an immutable reference to a mutable thing. Like a database connection. Or a connection to a remote microservice. Or an ETS table.