News for hackers. Stuff that matters.
Re:Claude Fable 5.2 and Mythos 5.2 by rfgplk on Monday November 09, 2026 @02:38PM (#50430692)
Every single time I click on these announcements the frontier model ALWAYS scores around ~60-80%. Every. Single. Time. Like, I just went back to the Opus 4.6 + 5 announcements, they were also scoring 60-80%.

https://www.anthropic.com/news/claude-opus-4-6 https://www.anthropic.com/news/claude-opus-5 https://openai.com/index/introducing-gpt-5-3-codex/

For every single announcement, not only do the top models always score 60-80%, the older models somehow start scoring 20-30% (even though previously they were scoring 60%). It's completely unclear whether the methodology changes, how these scores are even evaluated or which domains are being tested. What's worse is that the "benchmarks" keep being updated with no explanation or due process.

Re:Claude Fable 5.2 and Mythos 5.2 by wrs on Monday November 09, 2026 @03:10PM (#50431229)
The benchmarks have a lot of churn because the models keep saturating them. But the Anthropic announcements you linked, and the current one, have the benchmark versions listed right there in the table.
Re:Claude Fable 5.2 and Mythos 5.2 by mr_coffee on Monday November 09, 2026 @03:31PM (#50431603)
A real shell game. They keep changing benchmarks
Re:Claude Fable 5.2 and Mythos 5.2 by Bolwin on Monday November 09, 2026 @03:54PM (#50431982)
There is plenty of explanation, if you actually go seek out the benchmark. Past 80 benchmarks start approaching saturation and are not very useful to show differences