News for hackers. Stuff that matters.
Re:GPT-5.4 by dom96 on Thursday March 05, 2026 @02:08PM (#47265853)
Why do none of the benchmarks test for hallucinations?
Re:GPT-5.4 by tedsanders on Thursday March 05, 2026 @03:34PM (#47266925)
In the text, we did share one hallucination benchmark: Claim-level errors fell by 33% and responses with an error fell by 18%, on a set of error-prone ChatGPT prompts we collected (though of course the rate will vary a lot across different types of prompts).

Hallucinations are the #1 problem with language models and we are working hard to keep bringing the rate down.

(I work at OpenAI.)

Re:GPT-5.4 by netule on Thursday March 05, 2026 @02:51PM (#47266399)
Optics. It would be inconvenient for marketing, so they leave those stats to third parties to figure out.