Re:Details emerge of surprise board coup that ousted CEO Sam...
by Chamix on Saturday November 18, 2023 @02:59PM
(#38323717)
You are conflating Illya's belief in the transformer architecture (with tweaks/compute optimizations) being sufficient for AGI with that of LLMs being sufficient to express human-like intelligence. Multi-modality (and the swath of new training data it unlocks) is clearly a key component of creating AGI if we watch Sutskever's interviews from the past year.
Re:Details emerge of surprise board coup that ousted CEO Sam...
by cratermoon on Saturday November 18, 2023 @03:48PM
(#38324232)
Yes, I read "Attention Is All You Need", and I understand that the multi-head generative pre-trained model talks about "tokens" rather than language specifically. So in this case, I'm using "LLM" as shorthand for what OpenAI is doing with GPTs. I'll try to be more precise in the future.
That still leaves disagreement between Altman and Sutskever over whether or not the current technology will lead to AGI or "superintelligence", with Altman clearly turning towards skepticism.
Re:Details emerge of surprise board coup that ousted CEO Sam...
by Chamix on Saturday November 18, 2023 @07:10PM
(#38326553)
Fair enough, shame "Large Tokenized Models" etc never entered the nomenclature.
Re:Details emerge of surprise board coup that ousted CEO Sam...
by cratermoon on Sunday November 19, 2023 @12:12PM
(#38334777)
Some terms I've seen used for the technology:
Big-Data Statistical Models
Stochastic Parrots or parrot-tech
plausible sentence generators
glorified auto-complete
cleverbot
"a Blurry JPEG of the Web" <https://www.newyorker.com/tech/annals-of-technology/chatgpt-...>
and just plain ol' "machine learning"
Ilya Sutskever is a True Believer in LLMs being AGI, in that respect aligned with Geoff Hinton, his academic advisor at University of Toronto. Hinton has said "So by training something to be really good at predicting the next word, you’re actually forcing it to understand. Yes, it’s ‘autocomplete’—but you didn’t think through what it means to have a really good autocomplete"[1].
Meanwhile, Altman has decided that LLMs aren't the way.[2]
So Altman was pushing to turn the LLM into a for-profit product, to get what value it has, while the Sutskever-aligned faction thinks it is AGI, and want to keep it not-for-profit.
There's also some difference about whether or not AGI poses an "existential risk" or if the risks of current efforts at AI are along the lines of algorithmic bias, socioeconomic inequality, mis/disinformation, and techno-solutionism.
1. https://www.newyorker.com/magazine/2023/11/20/geoffrey-hinto...
2. https://www.thestreet.com/technology/openai-ceo-sam-altman-s...