Llama.cpp supports Vulkan. why doesn't Ollama?

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by SkyPuncher on Friday January 31, 2025 @07:56AM (#42887192)

Ollama - `brew install ollama`

llama.cpp - Read the docs, with loads of information and unclear use cases. Question if it has API compatibility and secondary features that a bunch of tools expect. Decide it's not worth your effort when `ollama` is already running by the time you've read the docs

[ Parent | on HN ]

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by singularity2001 on Friday January 31, 2025 @08:56AM (#42887677)

ollama run deepseek-r1:14b

[ Parent | on HN ]

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by kgwgk on Friday January 31, 2025 @09:02AM (#42887744)

https://formulae.brew.sh/formula/llama.cpp

[ Parent | on HN ]

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by LorenDB on Friday January 31, 2025 @09:03AM (#42887764)

Additionally, Ollama makes model installation a single command. With llama.cpp, you have to download the raw models from Huggingface and handle storage for them yourself.

[ Parent | on HN ]

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by trissi1996 on Friday January 31, 2025 @09:37AM (#42888089)

Not really, llama.cpp can download for quite some time, not as elegant as ollama but:

    llama-server --model-url "https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF/resolve/main/DeepSeek-R1-Distill-Qwen-32B-IQ4_XS.gguf"

Will get you up and running in one single command.

[ Parent | on HN ]

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by yencabulator on Friday January 31, 2025 @11:04AM (#42888868)

And now you need a server per model? Ollama loads models on-demand, and terminates them after idle, all accessible over the same HTTP API.

[ Parent | on HN ]