News for hackers. Stuff that matters.
Re:Show HN: Nomadic – Minimize RAG Hallucinations with 1 Hyp... by varunkrishnan17 on Thursday September 05, 2024 @07:59PM (#41461505)
Thanks for the well-thought out question Jadiker!

This is a potential limitation of N-gram precision with context matching, which we were using in the RAG demo for simplicity (though even with this, I don't think it would be so extreme :-) )

We already offer two other different hallucination detection approaches which should mitigate this problem - an LLM-as-a-judge model for evaluation, and semantic similarity matching. We've also considered, for example, using metrics such as BertScore. Do you have other ideas? :-)

There are no comments on this comment yet.