News for hackers. Stuff that matters.
Re:Gary Marcus's Kafkaesque critique of GPT-3 by gwern on Monday September 07, 2020 @03:56PM (#24402222)
Yes, pretty much. As Marcus admits in his writeup, even despite using doubly adversarially filtered GPT-3 errors, the final errors are due mostly to bad (no) prompting and then penalizing GPT-3 for not telepathically reading his mind in order to provide a serious response to a silly question.

I'm about halfway through the first 25 errors or so (it's very tedious so I've been putting it off), and as expected, GPT-3 is often correct zero-shot if you use even a trivial short prompt like 'This is a series of scenarios describing a human taking actions in the world, designed to test physical and common-sense reasoning.'

Re:Gary Marcus's Kafkaesque critique of GPT-3 by gwern on Monday September 07, 2020 @07:16PM (#24403735)
The point about prompting also applies to the GPT-2 counterexamples Marcus published earlier this year, and conspicuously fails to discuss in this one, BTW.