News for hackers. Stuff that matters.
Re:AI benchmarks are a bad joke – and LLM makers are the one... by ACCount37 on Saturday November 08, 2025 @12:17PM (#45858265)
And even "long term performance" splits itself into "performance on multi-turn instruction following" and "performance on agentic tasks" down the line. And "performance on agentic tasks" is a hydra in itself.

Capturing LLM performance with a single metric is a hopeless task. But even a single flawed metric beats no metrics at all.

There are no comments on this comment yet.