News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by world2vec on Friday March 31, 2023 @05:47PM (#35394107)
>I'm glad you're happy with the fact that LLaMA 30B (a 20gb file) can be evaluated with only 4gb of memory usage!

Isn't LLaMA 30B a set of 4 files (60,59Gb)?

-edit- nvm, It's quantized. My bad

[ Parent | on HN ]

There are no comments on this comment yet.