Re:Using mmap to make LLaMA load faster
by saurik on Wednesday April 05, 2023 @01:24PM
(#35457545)
Justine knows this and it is stated right there on the page:
> The first time you load a model after rebooting your computer, it's still going to go slow, because it has to load the weights from disk. However each time it's loaded afterwards, it should be fast (at least until memory pressure causes your file cache to be evicted).
> The first time you load a model after rebooting your computer, it's still going to go slow, because it has to load the weights from disk. However each time it's loaded afterwards, it should be fast (at least until memory pressure causes your file cache to be evicted).