Re:An understanding of AI’s limitations is starting to sink in
by HarHarVeryFunny on Tuesday June 16, 2020 @11:52AM
(#23540628)
Well, GPT-3 isn't any kind of general intelligence - it's explicitly architected as a language model - something that learns to pay attention to prior context to predict what word comes next. The only kind of world model it has is a statistical model of what word is most likely to come next based on the corpus it has been trained on.
You could argue that general intelligence is also based on prediction, and that a human's world model therefore isn't so different in nature, but there are some very significant differences ...
1) GPT-3's model is based only on a corpus of text (facts, lies, errors, etc) it was fed... there is no grounding in reality.
2) GPT-3 is only a passive model - it's not an agent that can act or in any way attempt to validate or augment it's world model.
3) GPT-3 is architecturally a language model .. it can get better with better or more data, but it's never going to be more than a language model.
The difference between a 3-year old's brain and GPT-3 is that the 3-year old's brain is not a one-trick pony ... it's a complex cognitive architecture, honed by millions of years evolution, capable of performing a whole range of tasks, not just language modelling.
The 3-year old's brain also has the massive advantage of being embedded in a 3-year old autonomous agent able to explore and interact with the world it's world model is representing... It you tell GPT-3 pigs can fly then as far as it is concerned pigs can fly, whereas the 3-year old can go seek out pigs and see that, in fact, they can't.
You could argue that general intelligence is also based on prediction, and that a human's world model therefore isn't so different in nature, but there are some very significant differences ...
1) GPT-3's model is based only on a corpus of text (facts, lies, errors, etc) it was fed... there is no grounding in reality.
2) GPT-3 is only a passive model - it's not an agent that can act or in any way attempt to validate or augment it's world model.
3) GPT-3 is architecturally a language model .. it can get better with better or more data, but it's never going to be more than a language model.
The difference between a 3-year old's brain and GPT-3 is that the 3-year old's brain is not a one-trick pony ... it's a complex cognitive architecture, honed by millions of years evolution, capable of performing a whole range of tasks, not just language modelling.
The 3-year old's brain also has the massive advantage of being embedded in a 3-year old autonomous agent able to explore and interact with the world it's world model is representing... It you tell GPT-3 pigs can fly then as far as it is concerned pigs can fly, whereas the 3-year old can go seek out pigs and see that, in fact, they can't.