News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now.

Re:Llama.cpp 30B runs with only 6GB of RAM now by m3kw9 on Friday March 31, 2023 @07:02PM (#35394943)
Looks like we can run OpenAI gpt3.5 with 32g ram?
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by sp332 on Friday March 31, 2023 @08:32PM (#35395700)
Where are you going to download that from?
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by operatingthetan on Saturday April 01, 2023 @02:24AM (#35397699)
Hey if Meta can see a leak, then a former non-profit like OpenAI definitely can.
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by llui85 on Saturday April 01, 2023 @05:57AM (#35398897)
Difference is that Meta was "publishing" the model to researchers, and one of them leaked it. I suspect OpenAI is a little less likely to share the GPT models around unfortunately...
[ Parent | on HN ]