Re:How Does GPT Obtain Its Ability? Tracing Emergent Abiliti...
by benreesman on Thursday December 15, 2022 @06:02AM
(#33997742)
I hope my comment didn’t imply that I’m anything short of impressed: it’s truly a feat.
But it’s one thing to regurgitate a causal structure that’s been said over and over and quite another to propose a novel and falsifiable hypothesis.
Telling a novel-ish just so story is well within the reach of a precocious 8 year old, and that’s selling the bright year old short.
Let’s see one of things propose a novel hypothesis before we get our drawers moist.
Sayings that it just looks like P(text|internet) is a tautology, it's a text predictor trained on the internet. This doesn't tell you anything about why phenomena like the above occur, or why it occurs only in large language models and only in some of them.