News for hackers. Stuff that matters.
Re:OpenAI GPT-4 vs. Groq Mistral-8x7B by simion314 on Friday March 22, 2024 @06:44AM (#39789311)
The problem is asking for facts, LLM are not a database so they know stuff but it is compressed so expect wrong facts, wrong names, dates, wrong anything.

We will need an LLM as a front end then it will generate a query to fetch the facts from the internet or a database , then maybe format the facts for your consumption.

Re:OpenAI GPT-4 vs. Groq Mistral-8x7B by ch_sm on Friday March 22, 2024 @07:29AM (#39789595)
That‘s exactly it. It‘s ok for LLMs to not know everything, because they _should_ have a means to look up information. What are some projects where this obvious approach is implemented/tried?
Re:OpenAI GPT-4 vs. Groq Mistral-8x7B by Jensson on Friday March 22, 2024 @07:50AM (#39789715)
But then you need an LLM that can separate between grammar and facts. Current LLMs doesn't know the difference, that is the main source to these issues, these models treat facts like grammar and that worked well enough to excite people but probably wont get us to a good state.
Re:OpenAI GPT-4 vs. Groq Mistral-8x7B by samus on Friday March 22, 2024 @10:20AM (#39791011)
This is called Retrieval Augmented Generation (RAG). The LLM driver recognizes a query, it gets send to a vector database or to an external system (could be another LLM...) and the answer is placed in the context. It's a common strategy to work around their limited context length, but it tends to be brittle. Look for survey papers.