News for hackers. Stuff that matters.

A comment on Llama.cpp 30B runs with only 6GB of RAM now. Parent comment

Re:Llama.cpp 30B runs with only 6GB of RAM now by conradev on Friday March 31, 2023 @06:15PM (#35394430)
Very cool! Are you testing after a reboot / with an empty page cache?
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by jart on Friday March 31, 2023 @06:38PM (#35394709)
* * *
[ Parent | on HN ]
Re:Llama.cpp 30B runs with only 6GB of RAM now by dekhn on Saturday April 01, 2023 @11:01AM (#35400835)
Linux has a command to drop caches at runtime (https://www.tecmint.com/clear-ram-memory-cache-buffer-and-sw...) which is VERY useful during debugging.
[ Parent | on HN ]