Re:GPT-3 has no idea what it’s talking about
by hackinthebochs on Saturday August 22, 2020 @05:41PM
(#24247492)
I'm not seeing an argument here. Mammalian evolution is great. So is backprop. They're both methods to efficiently search the state-space of parameters. If your argument is scale, you have to argue how scale precludes a sufficiently general learning algorithm and enough computation from capturing the underlying structure.
>The biochemistry of a single neuron is likely oodles more complex than the most complex AI we've made.
There's a lot of biochemistry in service to supporting the neurons existence exclusive from its functional properties for the brain. In terms of a neurons I/O mapping, deep learning has that covered[1]
>Yes, it's basically all just statistics... but we still don't understand how to go from statistics to intelligence.
But this assumes intelligence isn't an emergent phenomenon of sufficiently general learning. GPT-3 suggests this might be the case.
[1] https://www.biorxiv.org/content/10.1101/613141v1.full.pdf
It's a matter of scale. Adult human brains aren't just trained for 18 years with a static dataset - they've got hundreds of millions of years of mammalian evolution and fault tolerance built into them. Our brains self-reconfigure in response to external stimuli as we age, to the point where we can (in rare cases) lose significant fractions without becoming a vegetable. The biochemistry of a single neuron is likely oodles more complex than the most complex AI we've made.
It's like going from an analysis of a small family to an analysis of a civilization: at some point the sample size crosses a threshold and emergent phenomena start to dominate the system. Yes, it's basically all just statistics (ignoring quantum hocus pocus and the supernatural) but we still don't understand how to go from statistics to intelligence.