News for hackers. Stuff that matters.

A comment on Llama.cpp: Full CUDA GPU Acceleration. Parent comment

Re:Llama.cpp: Full CUDA GPU Acceleration by fisf on Tuesday June 13, 2023 @07:10AM (#36308419)
> Have you ever tried to iterate developing any serious class of algorithms in C++?

Yes, and it's not so bad. A lot of ML deployments have been based on C/C++ for inference anyway (with Python driving the training). So that's really nothing new. I.e. most Python research code is not deployable in terms of quality / performance.

[ Parent | on HN ]
Re:Llama.cpp: Full CUDA GPU Acceleration by nologic01 on Tuesday June 13, 2023 @07:42AM (#36308752)
> Yes, and it's not so bad.

In the absence of alternatives its workable. Libraries like eigen or armadillo help a lot. But Python with numpy has been extremely popular for a good reason.

[ Parent | on HN ]