News for hackers. Stuff that matters.

A comment on Llama.cpp supports Vulkan. why doesn't Ollama?. Parent comment

Re:Llama.cpp supports Vulkan. why doesn't Ollama? by nicce on Friday January 31, 2025 @07:48AM (#42887137)
> but VMs are pretty easy so just went that route.

Don’t you need at least 2 GPUs in that case and put kernel level passthrough?

[ Parent | on HN ]
Re:Llama.cpp supports Vulkan. why doesn't Ollama? by a12k on Friday January 31, 2025 @07:49AM (#42887148)
I don’t use GPU. Works fine, but the large Mixtral models are slow.
[ Parent | on HN ]
Re:Llama.cpp supports Vulkan. why doesn't Ollama? by bdhcuidbebe on Friday January 31, 2025 @08:37AM (#42887514)
i pass through my dGPU to VM and use iGPU for desktop
[ Parent | on HN ]