News for hackers. Stuff that matters.
Re:Experimenting with Local LLMs on macOS by a-dub on Monday September 08, 2025 @12:25PM (#45170261)
ollama is another good choice for this purpose. it's essentially a wrapper around llamacpp that adds easy downloading and management of running instances. it's great! also works on linux!
Re:Experimenting with Local LLMs on macOS by frontsideair on Tuesday September 09, 2025 @02:37AM (#45178257)
Ollama adding a paid cloud version made me postpone this post for a few weeks at least. I don't object them to make money, but it was hard to recommend a tool for local usage and make the first instruction to go to settings and enable airplane mode.

Luckily llama.cpp has come a long way and was at a point that I could easily recommend as the open source option instead.