News for hackers. Stuff that matters.

A comment on Llamafile lets you distribute and run LLMs with a single file. Parent comment

Re:Llamafile lets you distribute and run LLMs with a single ... by coldtea on Wednesday November 29, 2023 @08:06PM (#38467841)
>I make a small macOS app [1] which runs llama.cpp with a SwiftUI front-end. For the first version of the app I was obsessed with the single download -> chat flow and making 0 network connections. I bundled a model with the app and you could just download, open, and start using it. Easy! But as soon as I wanted to release a UI update to my TestFlight beta testers, I was causing them to download another 3GB. All 3 users complained :).

Well, that's on the MAS/TestFlight for not doing delta updates.

[ Parent | on HN ]
Re:Llamafile lets you distribute and run LLMs with a single ... by rgbrgb on Wednesday November 29, 2023 @10:20PM (#38469112)
Yes, though it does seem to be working for them. They have a special feature for lazy loading large assets but I opted for a simpler to me option (giving users a button to download a model if they don’t have one locally they want to use).
[ Parent | on HN ]