Re:Llamafile lets you distribute and run LLMs with a single ...
by simonw on Thursday November 30, 2023 @08:08AM
(#38473005)
If you want to use Docker then you can go ahead - I don't see anyone claiming that this brand new, highly experimental project should be used by everyone instead of Docker.
There are tradeoffs hers. For some people the tradeoff of a single executable file with everything in it compared to setting up a Docker system is worthwhile.
Re:Llamafile lets you distribute and run LLMs with a single ...
by reisse on Thursday November 30, 2023 @08:11AM
(#38473037)
Sure. I just question why people want to use both simultaneously.
Re:Llamafile lets you distribute and run LLMs with a single ...
by xylophile on Thursday November 30, 2023 @04:46PM
(#38479650)
Do you routinely download unsigned binaries of unprovable provenance and run them? Because if you do, you might eventually find reason to appreciate the additional isolation that namespaces et al give you very conveniently via Docker (or your favorite alternative).
Re:Llamafile lets you distribute and run LLMs with a single ...
by badrequest on Thursday November 30, 2023 @02:18PM
(#38477846)
`docker system prune --force --all --volumes` and they're gone.
Cargo cult is not in using Docker, but in using Docker to wrap something already wrapped into a comparable layer of abstraction.
Besides,
> not polluting my OS (filesystem and processes) with bits and bobs I downloaded off the internet
is purely self-deception. It's not like Docker images are not stored in some folder deep in the filesystem. If anything, it's harder to clean up after Docker than just doing rm -rf on a directory with llamafiles.