News for hackers. Stuff that matters.
Re:I Run LLMs Locally by throwaway314155 on Sunday December 29, 2024 @03:52PM (#42542975)
Open WebUI sure does pull in a lot of dependencies... Do I really need all of langchain, pytorch, and plenty others for what is advertised as a _frontend_?

Does anyone know of a lighter/minimalist version?

Re:I Run LLMs Locally by noman-land on Sunday December 29, 2024 @03:58PM (#42543036)
Re:I Run LLMs Locally by throwaway314155 on Sunday December 29, 2024 @04:20PM (#42543262)
I love what llamafile is doing, but I'm primarily interested in a frontend for ollama, as I prefer their method of model/weights distribution. Unless I'm wrong, llamafile serves as both the frontend and backend.
Re:I Run LLMs Locally by DrBenCarson on Sunday December 29, 2024 @05:13PM (#42543849)
LM Studio?
Re:I Run LLMs Locally by d3VwsX on Monday December 30, 2024 @04:48PM (#42553926)
If I understand the distinction correctly, I run llamafile as a backend. I start it with the filename of a model on the command-line (might need a -M flag or something) and it will start up a chat-prompt for interaction in the terminal but also opens a port that speaks some protocol that I can connect to using a frontend (in my case usually gptel in emacs).
Re:I Run LLMs Locally by figmert on Sunday December 29, 2024 @04:51PM (#42543635)
I ran it temporarily in Docker the other day. It was a 8gb image. I'm unsure why a webui is 8gb.
Re:I Run LLMs Locally by lolinder on Sunday December 29, 2024 @05:11PM (#42543836)
Some of the features (RAG retrieval) now use embeddings that are calculated in Open WebUI rather than in Ollama or another backend. It does seem like it'd be nice for them to refactor to make things like that optional for those who want a simpler interface, but then again, there are plenty of other lighter-weight options.
Re:I Run LLMs Locally by vunderba on Sunday December 29, 2024 @06:39PM (#42544638)
Given how much data (personal and otherwise) that you're likely going to feeding into it, I would HIGHLY recommend using an open-source chat interface like Jan.

It's cross platform, works with most LLMs, and has extension support.

https://github.com/janhq/jan

Another alternative is LibreChat, it's more fully featured but its heavier weight and spins up quite a few docker instances.

https://github.com/danny-avila/LibreChat

If you're going through the trouble of running models locally, it doesn't make sense to couple that with a proprietary closed source interface (like LM Studio).

Re:I Run LLMs Locally by synapsomorphy on Sunday December 29, 2024 @07:09PM (#42544884)
Did you misread? No one mentioned LM studio, open-webui (frontend) is open source and usually uses ollama (backend) which is also open source.

Jan and Librechat are both less popular than either open-webui or ollama, how do their features compare?

Re:I Run LLMs Locally by throwaway314155 on Sunday December 29, 2024 @07:30PM (#42545040)
No one had suggested LM Studio at this point in the thread.
Re:I Run LLMs Locally by ein0p on Sunday December 29, 2024 @06:44PM (#42544682)
Do you really care? Open WebUI has excellent UI and features, and its complexity is well hidden, IMO.
Re:I Run LLMs Locally by throwaway314155 on Sunday December 29, 2024 @06:51PM (#42544737)
I agree it's a fine interface, clearly a lot of work has gone into making it polished and feature-rich. I'm simply seeking alternatives. If there aren't any with the same feature set, I'll probably just use Open WebUI.
Re:I Run LLMs Locally by foldl2022 on Sunday December 29, 2024 @08:32PM (#42545478)
Off the topic: I am also in favor of minimalist.

In the case of AlphaGeometry, I made AlphaGeometryRE to get rid of tensorflow/jax/flax, and 100+ Python packages.

Re:I Run LLMs Locally by xyc on Monday December 30, 2024 @06:47PM (#42554904)