> On the other hand this thing got 83% on a test I got 47% on

Easy to do when it can memorize the answers in its training data and didn't get drunk while reviewing the textbook (that last part might just be me).

The Olympiad questions are puzzles, so you can't memorise the answers. To do well you need to both remember the foundations and exercise reasoning. They are written to be slightly novel to test this and not the same every year.

On the other other hand, it had the perfect recall of the collective knowledge of mankind at its metaphorical fingertips.

Is that supposed to make it sound more powerful or less powerful than it really is?

This thing also hallucinated a test directly into a function when I asked it to use a different data structure, which is not something I ever recall doing during all my years of tests and schooling.

Must have been quite the hangover to prevent your recalling this.