News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by lostmsu on Saturday April 01, 2023 @10:03AM (#35400396)
In fact, you can mmap from PyTorch directly.
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by ok123456 on Saturday April 01, 2023 @12:11PM (#35401406)
and numpy
[ Parent | on HN ]