When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation

Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by hiddencost on Tuesday August 04, 2026 @01:19PM (#49171924)

Weird moment for this take. We're seeing some of the fastest and most impressive progress ever right now.

Frontier labs have categorically different & better set ups for evaluation, they're fine. It's work but it's not a crisis.

[ Parent | on HN ]

Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by malfist on Tuesday August 04, 2026 @01:48PM (#49172306)

I've heard that every week of every month for the past three years. And yet, ask an LLM about a seahorse emoji and see what happens.

[ Parent | on HN ]

Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by CollinEMac on Tuesday August 04, 2026 @05:30PM (#49175451)

The seahorse emoji thing appears to be fixed actually. (Sonnet 5)

[ Parent | on HN ]

Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by ACCount37 on Tuesday August 04, 2026 @06:07PM (#49175894)

And it held true every week of every month for the past three years. AI progress is screaming forward at a breakneck pace.

[ Parent | on HN ]