News for hackers. Stuff that matters.
Re:Bringing K/V context quantisation to Ollama by vunderba on Thursday December 05, 2024 @10:54AM (#42329336)
As far as open source goes, I'd probably recommend LibreChat. It has connections for ollama, openai, anthropic, etc. It let's you setup auth so you can theoretically use it from anywhere (phone, etc.).

Fair warning, it's relatively heavyweight in so far as it has to spin up a number of docker instances but works very well.

https://github.com/danny-avila/LibreChat

There are no comments on this comment yet.