News for hackers. Stuff that matters.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by visarga on Thursday February 18, 2021 @01:03AM (#26176505)
Even so, memorisation implies word by word information retrieval and interpolation, it's not a hash table.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by jariel on Thursday February 18, 2021 @01:24AM (#26176612)
Yes, I mean, that's cool, but I'm just thinking that this isn't quite the AI we were thinking about before.

It's like a 'new form of storage and lookup' as opposed to the kind of 'magic algorithm' we usually think of when we think of AI. Or maybe that's just me.

Re:Google Open-Sources Trillion-Parameter AI Language Model ... by azinman2 on Thursday February 18, 2021 @01:32AM (#26176654)
Magic algorithms don’t exist. That’s the true reality of AI. I also felt the same thing and became dismayed as an undergrad at what was going on in computation having also studied neurobiology. But as it turns out, a shit ton of data with some stats can get you very, very far.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by jariel on Thursday February 18, 2021 @02:08AM (#26176887)
I didn't mean 'magic' (I know it's not that) - I just meant to imply that I think of AI as a 'function' not a 'lookup'.

Inputs -> Outputs not Search -> Response

Like if you train AI on a small dataset, it feels like what it's doing afterwords is a 'function' or 'algorithm' using what is in the end some arcane algebra.

But if you train on all the data in the world, with a trillion parameters ... well ... I kind of feel that 'all the data' is in an AI-style datastructure, that we are 'querying' with AI.

But that's just an observers abstraction.

Re:Google Open-Sources Trillion-Parameter AI Language Model ... by azinman2 on Thursday February 18, 2021 @05:24PM (#26186618)
See Searle’s Chinese room.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by feanaro on Thursday February 18, 2021 @02:35AM (#26177052)
Magic algorithms (or perhaps magic machines) do exist, though. See: humans.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by mlajtos on Thursday February 18, 2021 @04:18AM (#26177688)
We have ~150 trillion synapses. If every synapse was equivalent to just one parameter in ANN, then we could be just magic memoization machines.
Re:Google Open-Sources Trillion-Parameter AI Language Model ... by qayxc on Thursday February 18, 2021 @04:44AM (#26177875)
> If every synapse was equivalent to just one parameter in ANN, then we could be just magic memoization machines.

It's a poor comparison, though, since neurons and synapses aren't the same as parameters in a computational network. It's the same trap that news outlets routinely fall into when citing "storage capacity of the brain" in TBs and other such nonsense.