Disagreement Among Frontier LLMs on Real-World Fact-Checks

Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by kostaj on Thursday May 28, 2026 @11:13AM (#48310112)

Good point. Processing the substance of the answer might be too labor-consuming (1,000 claims x 5 models), but "thinking out loud" might improve the quality of the answers indeed. And we can still force/ask them to respond with a clear verdict at the end of their reasoning, as per the chosen rubric.

[ Parent | on HN ]

Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by airstrike on Thursday May 28, 2026 @11:24AM (#48310314)

If you have the model use a tool you can define the schema as a free text rationale field followed by one in the set of possible answers, so everything is nicely formatted as a JSON.

[ Parent | on HN ]

Re:Disagreement Among Frontier LLMs on Real-World Fact-Checks by kostaj on Thursday May 28, 2026 @12:14PM (#48311040)

Some models struggle combining JSON schema and web search capabilities.

[ Parent | on HN ]