News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by Miraste on Friday March 31, 2023 @06:47PM (#35394797)
This is incredible, great work. Have you tried it with the 65B model? Previously I didn't have a machine that could run it. I'd love to know the numbers on that one.
[ Parent | on HN ]

There are no comments on this comment yet.