Disagreement Among Frontier LLMs on Real-World Fact-Checks

Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by kostaj on Thursday May 28, 2026 @09:24AM (#48308638)

This paper covers only the disagreement between models and established only the floor of the error, based on the disagreement, but not which model is better. Planning to follow up with another study to benchmark against human-labelled verdicts still using a corpus that the models have not seen during training.

[ Parent | on HN ]

Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by aspenmartin on Thursday May 28, 2026 @10:38AM (#48309592)

You also need to involve better measures of agreement that are standard in the literature like krippendorfs alpha with ordinal metric. So many footguns in this methodology

[ Parent | on HN ]