News for hackers. Stuff that matters.
Re:Results of "Humanity's Last Exam" benchmark published by og_kalu on Thursday January 23, 2025 @02:08PM (#42806964)
This is true but that's because it's gotten hard to do much else. LLMs are eating up everything else that don't require long horizon planning or multimodality.

If you created a new benchmark today that didn't lean on the things I've mentioned or esoteric/super specialized domain knowledge (that would actually require some sort of super-human performance to ace) like this or Frontier Math, LLMs would probably do pretty well.

There are no comments on this comment yet.