But that isn’t claiming that it isn’t based on statistics, it’s just saying that the analysis being done isn’t shallow, which is clearly true
Re:AI knows things no one told it
by sharemywin on Friday May 12, 2023 @07:03PM
(#35923623)
I think to some extent it's using the attention to pay attention to which "statistics" to pay attention to.
> “It is certainly much more than a stochastic parrot, and it certainly builds some representation of the world—although I do not think that it is quite like how humans build an internal world model,” says Yoshua Bengio, an AI researcher at the University of Montreal.
> At a conference at New York University in March, philosopher Raphaël Millière of Columbia University [...] went a step further and showed that GPT can execute code, too, however. The philosopher typed in a program to calculate the 83rd number in the Fibonacci sequence. “It’s multistep reasoning of a very high degree,” he says. And the bot nailed it. When Millière asked directly for the 83rd Fibonacci number, however, GPT got it wrong: this suggests the system wasn’t just parroting the Internet. Rather it was performing its own calculations to reach the correct answer.
> This impromptu ability demonstrates that LLMs develop an internal complexity that goes well beyond a shallow statistical analysis. Researchers are finding that these systems seem to achieve genuine understanding of what they have learned.