News for hackers. Stuff that matters.
Re:GPT-5.6 by maxnevermind on Thursday July 09, 2026 @06:16PM (#48853103)
I'm surprised it is that low. Are not all top AI labs "cheating" and workaround LLMs's low sample efficiency by hiring people to generate more data points - similar problems with answers, so they can train models on those and improve scores? A good benchmark for general intelligence probably should be a complete black box, no sample data given/leaked at all.
Re:GPT-5.6 by Kuinox on Friday July 10, 2026 @03:36AM (#48856882)
Oh it's because the bench is lying. You need to pass each level without failing, if you fail a level, it count as you "lost" the minigame. The fact it start to get a score means it managed to get a 100% score on one of the minigame.