In the real world, it tends to perform better, though. Frankly, it also probably isn't as good as "mental math" as some humans, but that's also not what it's optimized for. However, it's able to perform roughly as well as an average human for practice SAT tests, so it doesn't seem that it really has general issues with intelligence test questions, even if they are more contrived than most real world problems. Nobody ever said that ChatGPT demonstrates intelligence similar to humans in every way, but again: who the hell cares? People are going to be saying this right into their own layoffs at this rate. I'm a career SWE, so I'm not really looking forward to it anymore than anyone else.

I also wonder: is there a chance you had more dialogue, but only presented this interaction?