News for hackers. Stuff that matters.
Re:Llamafile lets you distribute and run LLMs with a single ... by Zuiii on Wednesday November 29, 2023 @10:27PM (#38469165)
That's why I always download the original version and quantize myself. With enough swap, you can do it with a modest amount for ram. I never had to download a model twice.

But yes, unless there is a way to patch it, bundling the model with the executable like this is going to be more wasteful.

There are no comments on this comment yet.