Repeatedly, we’ve thought that humans and animals were different in kind, only to find that we’re actually just different in degree: elephants mourn their dead, dolphins have sex for pleasure, crows make tools (even tools out of multiple non-useful parts! [1]). That could be true here.
LLMs are impressive. Nobody knows whether they will or won’t lead to AGI (if we could even agree on a definition – there’s a lot of No True Scotsman in that conversation). My uneducated guess is that that you’re probably right: just continuing to scale LLMs without other advancements won’t get us there.
But I wish we were all more humble about this. There’s been a lot of interesting emergent behavior with these systems, and we just don’t know what will happen.
[1]: https://www.ox.ac.uk/news/2018-10-24-new-caledonian-crows-ca...
LLM proponents seem unwilling to accept that we comprehend the words we speak/write in a way that LLMs are not capable of doing.
Maybe their salary depends on them not understanding it.
I thought a forum of engineers would be more interested in the practical applications and possible future capabilities of LLMs, than in all these semantic arguments about whether something really is knowledge or really is art or really is perfect
Hard disagree. LLMs merely present the illusion of knowledge to the casual observer. A trivial cross examination usually is sufficient to pull back the curtain.
That effective theory is knowledge, literally.
People harping about “stochastic parrot” are just people repeating a shallow meme — ironically, like a stochastic parrot.
So what exactly is the usefulness of this discussion? You think "I'll trust my gut" is a useful argument in a debate?
FWIW my gut happens to agree with yours.
It hurts the pride of technical people that there's a revolution going on that they aren't involved in. Easier to just deny it or act like it's unimpressive.
How can I account for the cynicism that's so common on HN? It's got to be a psychological mechanism.
LLM models are very far off from humans in reasoning ability, but acting like most of the things humans do aren't just riffing on or repeating previous data is wrong, imo. As I've said before, humans been the stochastic parrots all along.
LLMs, irrespective of how powerful, are all subject to the fundamental limitation that they don't know anything. The stochastic parrot analogy remains applicable and will never be solved because of the underlying principles inherent to LLMs.
LLMs are not the pathway to AGI.