SafeGPT: New tool to detect LLMs' hallucinations, biases and privacy issues

Re:SafeGPT: New tool to detect LLMs' hallucinations, biases ... by crop_rotation on Friday April 21, 2023 @10:09AM (#35654012)

But GPT4 doesn't hallucinate on things which are popular enough to be replicated enough times on the web as knowledge. It hallucinates on things which are very less likely to be repeated many times. That rules out an external database with true answers. Unless the external database is supposed to contain all info queryable in all ways, in which case the database is just a better version of GPT-X.

The metamorphic testing approach is interesting and might work.

[ Parent | on HN ]

Re:SafeGPT: New tool to detect LLMs' hallucinations, biases ... by some_random on Friday April 21, 2023 @01:39PM (#35657316)

I've been playing with GPT4 summarization of hard knowledge that has an external database with true answers that GPT knows about, and it's still hallucinating regularly.

[ Parent | on HN ]