GPT-3 Is “Mindblowing” If You Don’t Question It Too Closely

Re:GPT-3 Is “Mindblowing” If You Don’t Question It Too Closely by wombatmobile on Sunday August 16, 2020 @03:16PM (#24180384)

Thank you for a considered response, SpicyLemonZest. When I posted that question I half expected to be howled down as a troll, a naysayer, or off-topic. I'm glad you did none of that and answered by sharing your thoughts. As humans in the community of HN, we are richer for it.

> most experts do seem to agree that GPT can't scale to general human-level intelligence.

What is it about human-level intelligence that's unreachable by machines, even machines as advanced as V42 of GPT, would you consider? Can you put your finger on it, or make a little start?

[ Parent | on HN ]

Re:GPT-3 Is “Mindblowing” If You Don’t Question It Too Closely by SpicyLemonZest on Sunday August 16, 2020 @03:36PM (#24180550)

I doubt there's any property of human-level intelligence that's inherently unreachable by machines. The question is whether GPT-42 would actually be tremendously advanced, or whether we're close to the limit of what the architecture can produce.

The consensus among experts I've read is that known architectural limitations make the GPT-n series unlikely to scale beyond "sounds very human if you don't question it too closely". Algebra, to pick a concrete example, is something that there's reason to believe the GPT-series can't learn effectively - the output generation mechanism is fundamentally incapable of looping or recursion, which severely limits how well it can break things down into subproblems.

[ Parent | on HN ]