News for hackers. Stuff that matters.

A comment on Disagreement Among Frontier LLMs on Real-World Fact-Checks. Parent comment

Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by kostaj on Thursday May 28, 2026 @09:46AM (#48308909)
Two of the five models used (Gemini+Search and Sonar Pro) have retrieval capabilities and used search when classifying the claims. The disagreement between them is still quite significant - 42%.
[ Parent | on HN ]
Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by simonw on Thursday May 28, 2026 @09:50AM (#48308954)
Here are those disagreements:

https://lite.datasette.io/?csv=https%3A%2F%2Fstatic.simonwil...

One example:

Researchers estimate that the average person ingests about 5 grams of plastic per week, which is approximately the weight of a credit card.

Gemini retrieval: Misleading

Sonar pro: Mostly True

[ Parent | on HN ]
Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by jeffbee on Thursday May 28, 2026 @10:24AM (#48309388)
Internally the statement is perfectly true: some researchers did estimate this, and the credit card is a fair proxy for a 5g mass.

Was the research flagrantly incorrect? Yes. But that does not affect the truth of the statement.

[ Parent | on HN ]