Re:Llama.cpp supports Vulkan. why doesn't Ollama?
by fkyoureadthedoc on Friday January 31, 2025 @08:31AM
(#42887444)
I can't install any .app on my fairly locked down work computer, but I can `brew install ollama`.
When I read the llama.cpp repo and see I have to build it, vs ollama where I just have to get it, the choice is already made.
I just want something I can quickly run and use with aider or mess around with. When I need to do real work I just use whatever OpenAI model we have running on Azure PTUs
Re:Llama.cpp supports Vulkan. why doesn't Ollama?
by kgwgk on Friday January 31, 2025 @09:04AM
(#42887779)
> I can `brew install ollama`.
Can you `brew install llama.cpp`?
Re:Llama.cpp supports Vulkan. why doesn't Ollama?
by fkyoureadthedoc on Friday January 31, 2025 @09:14AM
(#42887890)
probably, but why would I at this point? it's not on aider's supported list. If I needed to replace ollama for some reason I'd probably go with lms.
No this isn't. There are plenty of end user GUI apps that make it far easier than Ollama to download and run local LLMs (disclaimer: I build one of them). That's an entirely different market.
IMO, the intersection between the set of people who use a command line tool, and the set of people who are incapable of running `brew install llama.cpp` (or its Windows or Linux equivalents) is exceedingly small.