News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by zone411 on Friday March 31, 2023 @05:46PM (#35394097)
It really shouldn't act as a sparse model. I would bet on something being off.
[ Parent | on HN ]

There are no comments on this comment yet.