News for hackers. Stuff that matters.
Re:OpenLLM by tyre on Monday June 19, 2023 @12:36PM (#36393065)
You want the entire model to fit in memory. So if you’re looking at a download size of, say, 100GB then don’t run on less than that. Your machine will swap to/from disk constantly, which will be slow and wear out an SSD.

If you want to train a model, that’s a different story.

There are no comments on this comment yet.