Does the size of your llama matter? What's Ollama?, can I plug it to Chroma? (

Posted by torrmal on Monday September 25, 2023 @09:16PM
from the self-post dept.

If you have not seen it, this pretty cool open source project Ollama: It let's you run big LLMs in computers as small as your mac book!!

https://lu.ma/yourdatayourmodel

Ollama is launching support for Linux based machines this Wednesday at the SF AI Collective (come: https://lu.ma/yourdatayourmodel), alongside the founder of Chroma and a few more interesting people, join us in socializing and brainstorming about the future of all this!

( 3 comments | 3 points | on HN )

Re:Does the size of your llama matter? What's Ollama?, can I... by iain1992 on Monday September 25, 2023 @09:19PM (#37653114)

My Llama is huge! 70B parameters....Maybe I should try Ollama...

[ Parent | on HN ]

Re:Does the size of your llama matter? What's Ollama?, can I... by torrmal on Monday September 25, 2023 @09:20PM (#37653127)

well, I am not sure how small you can turn your 70b model into a small one, but its a great question for the Ollama creators: https://lu.ma/yourdatayourmodel

[ Parent | on HN ]

Re:Does the size of your llama matter? What's Ollama?, can I... by torrmal on Monday September 25, 2023 @09:27PM (#37653171)

actually, yes they do have support for Llama2 70B, you need 39GB of ram, compared to 130GB that it originally takes.

[ Parent | on HN ]