The ability to tell letter occurrence in a word is not a good measure for the quality of a large language model.

Anecdotally, it's less reliable when it comes to verifiable fact; but highly useful for brainstorming and sussing out creative solution.

Your results may vary.

The public school system in my country still teaches that the tongue has areas sensitive to specific tastes.

If a tutor tried to teach how to count letters instead of a useful skill I'd fire them.

Natural language is ambiguous and ChatGPT can handle that pretty well, if you want to count letters use Python or something.

That, and the thousands of numbskulls who waste their time trying to get ChatGPT to do math problems that a $1 calculator (or worst case, free Google Sheets) could do, is the most pointless waste of computing power, and I say this as someone who remembers Flash ads.

If you really are staring at strings and wanting those letters counted, you can easily ask it to write you a Python, JS, Ruby program that will do so, and in my experience for a task like that it will NAIL that task perfectly 100% of the time, and can even explain how it works to boot!

When I tested it on subjects I knew, it gave better answers than teachers in my high school (Poland, but I guess it might be similar elsewhere in the West)

Also, a friend physicist tested ChatGPT on the basics of quantum physics and he was surprised at how much it knew.

IMHO the chat is at a level of 1st year university student - of any subject available. The only issue being that it always sounds super confident, even if it's wrong.