GPT knows the letters each word is made up of, it can turn small snippets of text to Base64 reliably. The only reason it fails is that it is too stupid to understand the connection, not that it doesn't know what letters those words are made up of.

It knows the letters each word is made up when asked within a specific context. Humans also often struggle to recall things in one context that we have no problems with in another. It may well be reasonable to call it "too stupid" but at the same time it's unreasonable to call it stupid without then acknowledging that it can understand and reason. That it has gap in knowledge in areas we typically drill into young children and don't leave huge datasets online about is unsurprising to me.

That said, I incidentally think a whole lot of adults - even native English speakers - would struggle with the task given, and would repeatedly fail in the same way until given a detailed refresher.

E.g. being able to explain a rule and fail consistently to apply it is something I've seen up to and including supposed senior software engineers im interviews. Getting their mistakes explained and still repeating the same mistakes also.