News for hackers. Stuff that matters.
Re:Llama.cpp 30B runs with only 6GB of RAM now by d0mine on Saturday April 01, 2023 @06:11AM (#35398972)
It is easy to use mmap from Python. You can do zero copy too e.g., https://stackoverflow.com/questions/17244488/reading-struct-... (see mmap+frombuffer example)

Though in practice, in many cases, mmap won't be faster, it can be even slower than open+read.

There are no comments on this comment yet.