./llamafile-server-0.1 -m llama-2-13b.Q8_0.gguf
My experience is that the world of Python dependency management is a mess which sometimes works, and sometimes forces you to spend hours-to-days searching for obscure error messages and trying maybe-fixes posted in Github issues for some other package, just in case it helps. This sometimes extends further - e.g. with hours-to-days spent trying to install just-the-right-version-of-CUDA on Linux...
Anyway, the (somewhat annoying but understandable) solution that some developers take is to make their utility/app/whatever as self-contained as possible with a fresh install of everything from Python downwards inside a venv - which results in (for example) multiple copies of PyTorch spread around your HDD. This is great for less technical users who just need a minimal-difficulty install (as IME it works maybe 80-90% of the time), good for people who don't want to spend their time debugging incompatibilities between different library versions, but frustrating for the more technically-inclined user.
This is just another approach to the same problem, which presumably also presents an even-lower level of work for the maintainers, since it avoids Python installs and packages altogether?
What I suspect is happening is that people are cargo-culting zero-click installations. It seems rather fashionable right now.
"You can also also download just the llamafile software (without any weights included) from our releases page, or directly in your terminal or command prompt"
There is no cargo-culting going on. Some of us do legitimately appreciate it.
In this specific case there is an option for you that addresses this complaint, where the model isn't coupled with the app.
Is having everything normalized in your system that worth it? I would say having (some) duplicates in your system is mostly fine, better that having some spooky-action-at-a-distance break things when you don't expect.
I expect the future is something like Windows's WinSxS, NixOS's /nix/store, pnpm's .pnpm-store where that deduping isn't "online" but it still is somewhat automated and hidden from you.
Yes? It's right here, at the least.
And if that's the future, then the future sucks. We can teach people to be smarter, but no, instead our software has to bend over backwards to blow smoke up our ass because grandma.
I think you’re imposing your own very specific (and solvable!) gripe onto an unrelated project and it makes for pretty clumsy commentary.
But also: It might not be for a developer like you, but it is for a developer like me.
I enjoy writing software, but I don't particularly enjoy futzing with building things outside my day-to-day work, and on systems I don't write myself. If it was up to me everything would be one click.
Things like this are like accessibility: it benefits me even though I don't particularly need it.
I am utterly amazed and perplexed that answering ONE configuration question (which is all that is needed here) is too much. How lazy have we become?
One question. That is too much for folks?!
And I have no objection to the autodownloading. Just ask me where to put it first!
At least Makeayo asks where my models are now. It's obnoxious that I have to use symlinks for comfy/automatic....
All they need to do is ask me where my stuff is on first run, and an area in the config to update that setting. Not so hard!