News for hackers. Stuff that matters.
Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by totallygeeky on Tuesday August 04, 2026 @06:19PM (#49176016)
"Skill" with an LLM is nonsense. It's non-deterministic, so any equal efforts are not to be given equal outcome.
Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by whatisthiseven on Tuesday August 04, 2026 @06:40PM (#49176259)
You could say the same when applied to games with high RNG and chance, such as Slay the Spire 2, and yet those with real skill do perform far better than those without. Those with skill can clear the highest difficulties more often than those with lower skill.

Something being "non-deterministic" is orthogonal to whether or not skill plays a role.

Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by farbklang on Tuesday August 04, 2026 @06:31PM (#49176158)
not equal, but probabisticly better. Best example: give the agent a tight spec and it will perform better compared with a spec that leaves room for interpretation. This is true for all models, more or less. (purely anecdotal of course)
Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by AbrahamParangi on Tuesday August 04, 2026 @07:29PM (#49176688)
"Skill" at poker is nonsense. It's non-deterministic, so any equal efforts are not to be given equal outcome.
Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by sdenton4 on Tuesday August 04, 2026 @09:22PM (#49177478)
Baseball is also non-deterministic, and yet some players are apparently worth a lot more than others.
Re:When AI Benchmarks Plateau: A Systematic Study of Benchma... by aoeusnth1 on Wednesday August 05, 2026 @12:48PM (#49185405)
Soccer is nondeterministic. Are you as good as Messi?