What's the difference between acting as if it actually understands and "true understanding". I'd argue there is none, or at least that it doesn't matter. For instance there is nothing you could do to prove to me that you aren't just a black box acting on input in a sophisticated manner (eg. chinese room argument[1]), yet I give you the benefit of the doubt. GPT's lack of understanding of math may be a localized lack of understanding where it does "understand" other topics. I don't require you to display an understanding of quantum physics in order to prove that you're able to understand anything at all.

[1]: https://en.wikipedia.org/wiki/Chinese_room

I'm sure this topic is already the centre of much discussion, but from the sessions I've had with ChatGPT, it's quite obvious it doesn't really "understand" very much in the way humans do. At best it seems to understand what question you want an answer to, but it often fails miserably even in simple cases (try asking it how many letters certain words have, or to give examples of words ending in a particular letter etc.). But sure, eventually it may overcome those cases and make an excellent mimic of an intelligence with understanding. If it's genuinely able to produce output accurately emulating all the sorts of logical reasoning humans can do then it may well be impossible to distinguish it from "the real thing" (whatever that actually is...)

Humans understand and there is just no comparison. Can an AI make a major novel discovery as humans have? How could they it if they don’t understand language?