News for hackers. Stuff that matters.

A comment on Llama.cpp: Full CUDA GPU Acceleration. Parent comment

Re:Llama.cpp: Full CUDA GPU Acceleration by tryauuum on Tuesday June 13, 2023 @11:11AM (#36311401)
Would the problem "how can I run this cool ML project from GitHub" be solved if developers would publish their container images on dockerhub? The only downside I see is enormous image sizes
[ Parent | on HN ]
Re:Llama.cpp: Full CUDA GPU Acceleration by anotherhue on Tuesday June 13, 2023 @12:21PM (#36312521)
"Here's a massive glob of mystery-meat state to run my massive glob of mystery-meat tensors".
[ Parent | on HN ]
Re:Llama.cpp: Full CUDA GPU Acceleration by rvnx on Wednesday June 14, 2023 @04:23AM (#36323045)
"and by the way, inside the image I slightly modified one file of the tensors library (and this is undocumented) and this totally changes the output if the fix is not there"
[ Parent | on HN ]
Re:Llama.cpp: Full CUDA GPU Acceleration by hobo_in_library on Wednesday June 14, 2023 @02:10AM (#36322033)
They just need a dockerfile that builds correctly. No need to make an image available, the dockerfile should be able to build it consistently
[ Parent | on HN ]