News for hackers. Stuff that matters.

A comment on Show HN: Ollama for Linux – Run LLMs on Linux with GPU Acceleration. Parent comment

Re:Show HN: Ollama for Linux – Run LLMs on Linux with GPU Ac... by brucemacd on Tuesday September 26, 2023 @01:38PM (#37662918)
It could be possible to take a similar installer approach in the future for cuBLAS or hipBLAS for AMD GPUs, so there is hope.
[ Parent | on HN ]
Re:Show HN: Ollama for Linux – Run LLMs on Linux with GPU Ac... by brucethemoose2 on Tuesday September 26, 2023 @02:26PM (#37663703)
Koboldcpp does this: https://github.com/LostRuins/koboldcpp/releases/tag/v1.44.2

They basically just ship executables for different llama.cpp backends and select the correct one with a python script, which is fine, as the executables are really small.

[ Parent | on HN ]