Re:Using LLaMA with M1 Mac and Python 3.11
by orf on Sunday March 12, 2023 @04:03PM
(#35125481)
SSDs don’t wear from reading, only from writing.
Assuming a sensible, somewhat linear layout using mmap to map the weights would give you the ability to load a lot in memory, with potentially a fairly minimal page-in overhead
Assuming a sensible, somewhat linear layout using mmap to map the weights would give you the ability to load a lot in memory, with potentially a fairly minimal page-in overhead