Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by azinman2 on Thursday February 18, 2021 @01:32AM
(#26176654)
Magic algorithms don’t exist. That’s the true reality of AI. I also felt the same thing and became dismayed as an undergrad at what was going on in computation having also studied neurobiology. But as it turns out, a shit ton of data with some stats can get you very, very far.
Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by jariel on Thursday February 18, 2021 @02:08AM
(#26176887)
I didn't mean 'magic' (I know it's not that) - I just meant to imply that I think of AI as a 'function' not a 'lookup'.
Inputs -> Outputs not Search -> Response
Like if you train AI on a small dataset, it feels like what it's doing afterwords is a 'function' or 'algorithm' using what is in the end some arcane algebra.
But if you train on all the data in the world, with a trillion parameters ... well ... I kind of feel that 'all the data' is in an AI-style datastructure, that we are 'querying' with AI.
But that's just an observers abstraction.
Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by azinman2 on Thursday February 18, 2021 @05:24PM
(#26186618)
See Searle’s Chinese room.
Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by feanaro on Thursday February 18, 2021 @02:35AM
(#26177052)
Magic algorithms (or perhaps magic machines) do exist, though. See: humans.
Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by mlajtos on Thursday February 18, 2021 @04:18AM
(#26177688)
We have ~150 trillion synapses. If every synapse was equivalent to just one parameter in ANN, then we could be just magic memoization machines.
Re:Google Open-Sources Trillion-Parameter AI Language Model ...
by qayxc on Thursday February 18, 2021 @04:44AM
(#26177875)
> If every synapse was equivalent to just one parameter in ANN, then we could be just magic memoization machines.
It's a poor comparison, though, since neurons and synapses aren't the same as parameters in a computational network. It's the same trap that news outlets routinely fall into when citing "storage capacity of the brain" in TBs and other such nonsense.
It's like a 'new form of storage and lookup' as opposed to the kind of 'magic algorithm' we usually think of when we think of AI. Or maybe that's just me.