And finally gives up admitting it is a counterexample. Even saying "holy shit". That's what any human expert would do: presented with an extremely improbable fact, it would verify it, then in disbelief verify it again, then change method, then try yet another one, checking its understanding of the problem itself... and finally concede. Because this is what intelligence looks like- understanding the problem, its relevance, its context, self-doubting if the solution looks too simple to be true, double-checking all calculations and trying different approaches, etc.

I think Haiku still does this with the seahorse emoji. Doesn't feel like it's finding contradictory evidence, though, just feels like it's trying both options in case one works.

Precisely - except you can get it to do the same thing about the flat earth or faked moon landings or the unity of all being as will or just about anything else.