News for hackers. Stuff that matters.
Re:Llama.cpp supports Vulkan. why doesn't Ollama? by zozbot234 on Friday January 31, 2025 @09:17AM (#42887908)
> it's too slow to be useful with such specs.

Only if you insist on realtime output: if you're OK with posting your question to the model and letting it run overnight (or, for some shorter questions, over your lunch break) it's great. I believe that this use case can fit local-AI especially well.

There are no comments on this comment yet.