News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by sroussey on Saturday April 01, 2023 @01:32PM (#35402161)
On a Mac, mmap definitely works for the GPU since it’s all the same unified memory.
[ Parent | on HN ]

There are no comments on this comment yet.