News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by smaddox on Friday March 31, 2023 @06:55PM (#35394881)
Based on that discussion, it definitely sounds like some sort of bug is hiding. Perhaps run some evaluations to compare perplexity to the standard implementation?
[ Parent | on HN ]

There are no comments on this comment yet.