Re:Ask HN: Do you think Python will disappear for LLM infere...
by astronautas on Friday January 03, 2025 @01:00PM
(#42587919)
I agree Python is hardly replacable. Btw, I only mean inference, not training. Training imho should stay pure Python, you can achieve mega throughout with it for batch processing.
Re:Ask HN: Do you think Python will disappear for LLM infere...
by franktankbank on Friday January 03, 2025 @01:17PM
(#42588076)
> Btw, I only mean inference
D'oh! Yea that changes things. I would be considering UI integration from the inference side.
edit: you've removed a line while I was resonding like: "maybe go is a sweet-spot"
I think if you are looking to replace python as a runtime you'd maybe be better off arguing for safety, python is much more easily corruptible so if maybe you don't trust your machine(s) you are training on and don't want to be mislead by someone hacking your initializations then arguably running a compiled program with certain guarantees is safer.