News for hackers. Stuff that matters.
Re:Serving 70B-Scale LLMs Efficiently on Low-Resource Edge D... by yjftsjthsd-h on Thursday October 03, 2024 @12:07PM (#41732007)
I'm not aware of any Debian family distro that packages it, but NixOS has at least ollama and llama-cpp in its repos. Honestly even if the more stable distributions did have these things packaged, I would hesitate to use the packaged versions because all of this stuff is still so quickly moving that you'd be on an old version and it would hurt.

Edit: Arch has ollama in official repos too. OpenSUSE has https://software.opensuse.org/package/ollama .

There are no comments on this comment yet.