I would think the problem is that these LLMs don’t have a concept of any reality/objects outside of language. So many of the insights that come intuitively to humans, like Newton’s apple, will probably not come to an LLM. But I’m really just guessing. I also wouldn’t have thought that ChatGPT can draw a unicorn, and yet apparently it can.