Re:OpenLLM
by tyre on Monday June 19, 2023 @12:36PM
(#36393065)
You want the entire model to fit in memory. So if you’re looking at a download size of, say, 100GB then don’t run on less than that. Your machine will swap to/from disk constantly, which will be slow and wear out an SSD.
If you want to train a model, that’s a different story.
If you want to train a model, that’s a different story.