I just know that we don’t actually know. We assume we are special, but we could just be stringing words together based on what we experienced and learned with what we remember for prompting.

There's a lot we don't know about the details of how our brains work, but we do know that our brains are connected to the external world in a huge number of ways that LLMs are not. The only connections to the external world that LLMs have are their training data and the prompts they are given. That is an incredibly paltry level of connection compared with the connections that our brains have. Those connections make a huge difference.

As AI technology evolves, I would expect that the connections AIs have with the external world will become much more extensive, and that will make a huge difference as well. But current LLMs simply don't have that, and it shows.