News for hackers. Stuff that matters.

A comment on Dot: use of local LLMs and RAG in particular to interact with documents. Parent comment

Re:Dot: use of local LLMs and RAG in particular to interact ... by prosunpraiser on Tuesday April 09, 2024 @02:41AM (#39976718)
I haven’t seen on how it fares on uncensored use-cases, but from what I see Q5_K variants of Mistral 7B are not very far from Mixtral 8x7B (the latter requires 64GB of RAM which I don’t have).

Tried open-webui yesterday with Ollama for spinning up some of these. It’s pretty good.

[ Parent | on HN ]

There are no comments on this comment yet.