Eventually, all the better AGI tests should have large private evaluation datasets with no possible cheating or feedback loops. We're getting there.

Wasn’t that for human tests, i.e. not specifically AI benchmarks? Benchmarks should generally not be game-able by overfitting.

The article shows all the tests against human performance.

The math one in particular is the one where small variations reduce the success rate significantly. I can’t find the source but it was pasted here in the last 2 weeks.

You're probably remembering this: https://news.ycombinator.com/item?id=42565606