News for hackers. Stuff that matters.
Re:New LLM optimization technique slashes memory costs up to... by wint3rmute on Tuesday December 17, 2024 @03:48AM (#42439576)
Ollama was the easiest way to set up local LLMs for me.

https://ollama.com/

Re:New LLM optimization technique slashes memory costs up to... by solarkraft on Tuesday December 17, 2024 @09:05AM (#42441410)
With llama3.2:1b, llama3.2:3b and llama 3.1:8b being the main ones I tried and found impressive.