> But if you concoct a novel problem that isn't found anywhere on the internet and confront them with it, they fail absurdly.

Often they do, but sometimes they don't.

> Even if it's something that a kid could figure out.

Intelligent and smart aren't synonyms. Modern LLM's are obviously pretty stupid at times, but even a human with an IQ of 65 has some intelligence.