News for hackers. Stuff that matters.
Re:Claude Haiku 5.5 by chriddyp on Thursday October 08, 2026 @10:15AM (#50006081)
It's a valid concern. We're not releasing the dataset, the answers, or the full set of questions to help prevent this. At the same time, I like to share where it gets things wrong in a bit more detail, which involves sharing a bit of the exam. I expect that we'll create a new benchmark with a new dataset in 6 months.

There are no comments on this comment yet.