Re:RedPajama (Open LLaMA Replication) 3B and 7B models released
by Joeri on Saturday May 06, 2023 @05:47AM
(#35839753)
The critical factor seems to be the ability to fit the whole model in RAM (--mlock option in oobabooga). With Apple's RAM prices most M1/M2 owners probably don't have the 32 GB RAM required to fit a 4bit 30B model.
[ Parent
| on HN ]