But by that standard, I could just ask GPT to write code to multiply the numbers. It's not even trying you be an embodied agent with a hand able to type onto a calculator.

GPT is remarkable to me not for its failings, but because of how far it can get when it's only trying to win the game of "predict what token comes next".

To be more accurate, the 7 year old will tell you they can't do it and suggest the use of a calculator.

GPT simply gives you a complete hallucination. This can be gotten around imperfectly with prompt engineering to get GPT to admit it doesn't know. But still, hallucinations are pretty dangerous failure modes in many applications, and definitely something we don't want in a future iteration. I don't know how much of a change it is to create a GPT without hallucinations, but I suspect that it is non-trival.