News for hackers. Stuff that matters.
Re:Llamafile lets you distribute and run LLMs with a single ... by vsnf on Thursday November 30, 2023 @02:36AM (#38470777)
My attempt to run it with the my VS 2022 dev console and a newly downloaded CUDA installation ended in flames as the compilation stopped with "error limit reached", followed by it defaulting to a CPU run.

It does run on the CPU though, so at least that's pretty cool.

Re:Llamafile lets you distribute and run LLMs with a single ... by jart on Thursday November 30, 2023 @02:53AM (#38470862)
I've received a lot of good advice today on how we can potentially improve our Nvidia story so that nvcc doesn't need to be installed. With a little bit of luck, you'll have releases soon that get your GPU support working.