News for hackers. Stuff that matters.
Re:Llama.cpp: Full CUDA GPU Acceleration by dontreact on Tuesday June 13, 2023 @08:49AM (#36309470)
As far back as 2015, it’s been common to take neural net inference and get a C++ version of it. That’s what this is.

It didn’t make python obsolete then and it won’t now.

Training (orchestration, architecture definition etc.) and data munging (scraping cleaning analyzing etc.) are much easier with python than C++, and so there is no chance that C++ takes over as the lingua Franca of machine learning until those activities are rare

Re:Llama.cpp: Full CUDA GPU Acceleration by emmender on Tuesday June 13, 2023 @10:50AM (#36311048)
python is syntactic sugar - the heavy lifting is done by c/c++ bindings.

many ML experts are not software engineers. They just want syntax to get their job done. fair enough.