agree, it depends, always benchmark, but my question is rather generic i.e. I am looking for a perspective.

Ok my perspective of pursuing something like this in Rust and continuing the motherboard analogy, is that you take on the full burden of developing everything in your laptop onto a single chip (sounds pretty awesome really) but your boundaries can end up blurred as everything is just a single etch. Forcing boundaries by making components with hard won decisions creates a marketplace of exchangeable components. Force everyone to agree on that API by dropping across a hard language boundary.

can't disagree (a tradeoff).

It's getting very philosophical, without a (perhaps impossibly complex/fraught) model of real-world programmers using all available languages doing just what you ask there is no way to find the sweet-spot. I've given some rules of thumb why python still rules and why things in general favor standard forms easily reviewable by humans and peppered with magic only understandable by geeks.

edit: you've removed a line while I was resonding like: "maybe go is a sweet-spot"

I think if you are looking to replace python as a runtime you'd maybe be better off arguing for safety, python is much more easily corruptible so if maybe you don't trust your machine(s) you are training on and don't want to be mislead by someone hacking your initializations then arguably running a compiled program with certain guarantees is safer.

I agree Python is hardly replacable. Btw, I only mean inference, not training. Training imho should stay pure Python, you can achieve mega throughout with it for batch processing.

> Btw, I only mean inference

D'oh! Yea that changes things. I would be considering UI integration from the inference side.