News for hackers. Stuff that matters.
Re:AI benchmarks are a bad joke – and LLM makers are the one... by daveguy on Saturday November 08, 2025 @02:28PM (#45859232)
Because AI are good at devolving to the highest score, regardless of test intent. For most problems "ask_hooman", or especially the plural, would be much more effective. So, the degenerate case would dominate and tell you precisely zero about the intelligence of the AI. If a specific "tool" is more adept than the "AI" then "choose tool" will always be the correct answer. But I agree, a tight time constraint would help.

There are no comments on this comment yet.