Re:ChatGPT passes the 2022 AP Computer Science A free respon...
by kaba0 on Monday December 05, 2022 @02:40AM
(#33862666)
I don’t know, isn’t the safer bet is that scaling will eventually reach a dead end? I honestly fail to see how a language model could “execute” a sequence of reasoning steps, as it doesn’t think in a symbolic way. Do correct me if I’m wrong but it would require a complex rearchitecture, so not sure we are any closer, we just have a very impressive, smart search engine now.
Re:ChatGPT passes the 2022 AP Computer Science A free respon...
by akiselev on Monday December 05, 2022 @03:41AM
(#33863018)
It’s not just a safe bet but almost guaranteed. Humans combine their internal language models with physical intuition and experimentation from the moment they are born. There is zero chance that an AI can understand the physical world without access to it [1]. Until it has that access, it’s no more than a glorified context specific Markov chain generator
[1] Henceforth called Kiselev’s conjecture, a corollary of Moravec’s paradox: https://en.m.wikipedia.org/wiki/Moravec's_paradox
Re:ChatGPT passes the 2022 AP Computer Science A free respon...
by zone411 on Monday December 05, 2022 @04:42AM
(#33863422)
It's possible for models to learn a lot about everyday physics from videos.
Re:ChatGPT passes the 2022 AP Computer Science A free respon...
by akiselev on Monday December 05, 2022 @06:21AM
(#33864127)
No, it isn't. Not yet.
Re:ChatGPT passes the 2022 AP Computer Science A free respon...
by zone411 on Monday December 05, 2022 @01:01PM
(#33869149)
You said "There is zero chance that an AI can understand the physical world without access to it," which is wrong. It is possible. Using videos is an active research area, e.g. https://proceedings.neurips.cc/paper/2021/hash/07845cd9aefa6... or https://arxiv.org/abs/2205.01314
Re:ChatGPT passes the 2022 AP Computer Science A free respon...
by akiselev on Monday December 05, 2022 @11:43PM
(#33876016)
Thank you for the links, it's fascinating!
Fact is, without a feedback loop that can run physical experiments like infants do from the moment they're born, I highly doubt they will develop a useful intuition using just video. Hence the conjecture
I don't think so, the scaling laws haven't failed so far. I fully expect that making the model bigger and training it on more data will make it better at logic.
For a nice example with image models, Scott Alexander made a bet that newer image models would be able to do the things that Dall-E 2 gets wrong. [1] (This post also discusses how GPT-3 could do many things that GPT-2 got wrong.) He won the bet three months later through Imagen access. [2]
[1]: https://astralcodexten.substack.com/p/my-bet-ai-size-solves-... [2]: https://astralcodexten.substack.com/p/i-won-my-three-year-ai...