Re:Llama.cpp 30B runs with only 6GB of RAM now
by pritambaral on Saturday April 01, 2023 @03:20AM
(#35398012)
mmap-ed memory pages backed by a file that aren't dirty aren't counted in an process's RSS usage, only kernel page cache. The mmap-ed regions of virtual memory does get counted in VSZ (virtual memory) but that is just virtual and can be larger than RAM+swap.
[ Parent
| on HN ]
There are no comments on this comment yet.