ChatGPT is impressive, but gets many things wrong. If you know what you are doing it's an amazing programming assistant. It makes me noticeably more productive. It may lead someone who doesn't know what they are doing in weird rabbit holes that will lead nowhere however.
One silly example. I was using a library I hadn't use before, and I asked how I could get certain attributes. It gave me an answer that would't compile at all, the imports didn't exist.
Then when I mentioned that it didn't work, it game me a slightly different answer, that also didn't work, and explained that the previous answer was valid for 3.x. in 1.x or 2.x the new answer was the correct one.
But there's the catch. There's no version 3.x. there's not even a 2.x. It's language model just statically got to that conclusion.
Doesn't make it any less impressive to me. It gets things right often enough, or at least points me in a good direction. I effectively learned new things using it. But it can't replace a developer.
Using ChatGPT as if it was General AI is similar to eat a meal using a hammer and a screwdriver as utensils. You can probably do it, but nobody will have a good time.
https://abcnews.go.com/Technology/openai-ceo-sam-altman-ai-r...
I neither know how LLMs work nor how our brains work. And I don't know what could be parallel between these two. For my very very limited knowledge of how properties can emerge from unique arrangements of constituent components (the S-R latch giving rise to state - i.e. memory - comes to mind), I would not at this point write off the possibility that a very large / very deep / very intricate neural network trained on language prevalence in very very large datasets could manifest properties that we would interpret as reasoning.
And I further wouldn't write off the we humans may owe no small part of our reasoning ability to language comprehension that we begin to ascertain from infancy.