News for hackers. Stuff that matters.

A comment on Llama.cpp supports Vulkan. why doesn't Ollama?. Parent comment

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by trissi1996 on Friday January 31, 2025 @09:37AM (#42888089)
Not really, llama.cpp can download for quite some time, not as elegant as ollama but:

    llama-server --model-url "https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF/resolve/main/DeepSeek-R1-Distill-Qwen-32B-IQ4_XS.gguf"
Will get you up and running in one single command.
[ Parent | on HN ]
Re:Llama.cpp supports Vulkan. why doesn't Ollama? by yencabulator on Friday January 31, 2025 @11:04AM (#42888868)
And now you need a server per model? Ollama loads models on-demand, and terminates them after idle, all accessible over the same HTTP API.
[ Parent | on HN ]