News for hackers. Stuff that matters.

A comment on Does the size of your llama matter? What's Ollama?, can I plug it to Chroma? (.

Re:Does the size of your llama matter? What's Ollama?, can I... by iain1992 on Monday September 25, 2023 @09:19PM (#37653114)
My Llama is huge! 70B parameters....Maybe I should try Ollama...
[ Parent | on HN ]
Re:Does the size of your llama matter? What's Ollama?, can I... by torrmal on Monday September 25, 2023 @09:20PM (#37653127)
well, I am not sure how small you can turn your 70b model into a small one, but its a great question for the Ollama creators: https://lu.ma/yourdatayourmodel
[ Parent | on HN ]
Re:Does the size of your llama matter? What's Ollama?, can I... by torrmal on Monday September 25, 2023 @09:27PM (#37653171)
actually, yes they do have support for Llama2 70B, you need 39GB of ram, compared to 130GB that it originally takes.
[ Parent | on HN ]