Re:Llamafile lets you distribute and run LLMs with a single ...
by finnjohnsen2 on Thursday November 30, 2023 @04:37AM
(#38471541)
I see this as not polluting my OS (filesystem and processes) with bits and bobs I downloaded off the internet. The cargo cult is a clean, safe and warm space and I highly recommend it.
[ Parent
| on HN ]
Re:Llamafile lets you distribute and run LLMs with a single ...
by reisse on Thursday November 30, 2023 @07:36AM
(#38472727)
I see you and other commenters don't quite understand my point. If you're wrapping model into a docker container, you don't need amalgamated single file version. It makes it harder to upgrade llamafile/model weights separately afterwards, it needs you to store separate llamafile binary for each container, etc, etc. Why not just build proper layered image with separate layer for llama.cpp and separate layer or volume for model?
Cargo cult is not in using Docker, but in using Docker to wrap something already wrapped into a comparable layer of abstraction.
Besides,
> not polluting my OS (filesystem and processes) with bits and bobs I downloaded off the internet
is purely self-deception. It's not like Docker images are not stored in some folder deep in the filesystem. If anything, it's harder to clean up after Docker than just doing rm -rf on a directory with llamafiles.
Re:Llamafile lets you distribute and run LLMs with a single ...
by simonw on Thursday November 30, 2023 @08:08AM
(#38473005)
If you want to use Docker then you can go ahead - I don't see anyone claiming that this brand new, highly experimental project should be used by everyone instead of Docker.
There are tradeoffs hers. For some people the tradeoff of a single executable file with everything in it compared to setting up a Docker system is worthwhile.
Re:Llamafile lets you distribute and run LLMs with a single ...
by reisse on Thursday November 30, 2023 @08:11AM
(#38473037)
Sure. I just question why people want to use both simultaneously.
Re:Llamafile lets you distribute and run LLMs with a single ...
by xylophile on Thursday November 30, 2023 @04:46PM
(#38479650)
Do you routinely download unsigned binaries of unprovable provenance and run them? Because if you do, you might eventually find reason to appreciate the additional isolation that namespaces et al give you very conveniently via Docker (or your favorite alternative).
Re:Llamafile lets you distribute and run LLMs with a single ...
by badrequest on Thursday November 30, 2023 @02:18PM
(#38477846)
`docker system prune --force --all --volumes` and they're gone.