anon389r58r58
PINNs have serious problems with the way the "PDE-component" of the loss function needs to be posed, and outside of throwing tons of, often Chinese, PhD students, and postdocs at it, they usually don't work for actual problems. Mostly owed to the instabilities of higher order automatic derivatives, at which point PINN-people begin to go through a cascade of alternative approaches to obtain these higher-order derivatives. But these are all just hacks.
Emulating Python's ML ecosystem is not going to be a viable strategy. The investment into the Python-based standard is just too large.
What I could see happening though is that the continuous evolution of the ML ecosystem will further abstract components of the software stack down to an MLIR/LLVM abstraction level at which point something like Julia could also use this componentry. Sort of a continuum of components, where the "frontend" language and associated programming style is the remaining choice for the user to make.
And so the perennial cycle continues and another Julia AD-package emerges, and ignores all/most previous work in order to claim novelty.
Without a claim for a complete list: ReverseDiff.jl, ForwardDiff.jl, Zygote.jl, Enzyme.jl, Tangent.jl, Diffractor.jl, and many more whose name has disappeared in the short history of Julia...
I just don't think it makes sense to connect multiple of them into a "cluster" to work with bigger models, as the networking bandwidth isn't good enough and you'd have to fit multiple of these big boxes into your local space. Then I might as well put up a rack in a separate room.
The problem of the form factor will remain. The tinybox is 15U big for compute that you'd normally expect to find in a 4U form factor.
Because JAX is not designed around a mature compiler stack. The history of Jax is more so that it matured alongside the compiler...