Re:Teaching GPT-3 to Identify Nonsense
by yellowstuff on Wednesday July 29, 2020 @04:38PM
(#23991332)
This is a wild guess- When GPT-3 got confused by "no legs", I don't think it's because it would have any trouble correctly interpreting "no legs" in other contexts. In this context it was searching in a semantic space set up by "a head, a tail, is brown", and the general form of a question looking for an animal. GPT-3 didn't find anything in the semantic space matching "no legs" so it assumed that was an error and corrected it. It's actually fairly close to how a human would solve the riddle, except when the human noticed nothing matched the explicit and implicit prompts, they would correct the implicit request for an animal, and then reinterpret the rest of the riddle.
[ Parent
| on HN ]
Re:Teaching GPT-3 to Identify Nonsense
by red75prime on Thursday July 30, 2020 @11:41PM
(#24005426)
GPT-3 generates output word by word. And it doesn't make distinctions between prompt words and words it has generated. Generated words are selected somewhat randomly (if temperature >0). So "the train of GPT-3 thought" can be easily derailed by unfortunate random selection of a single word.
When it generated "I don't know what you mean by", it had to continue with something relevant. In this case "no legs" was selected.
In other words GPT-3 doesn't have machinery to stop and think "What I'm rambling about?".
Re:Teaching GPT-3 to Identify Nonsense
by LoSboccacc on Wednesday July 29, 2020 @06:21PM
(#23992228)
> a head, a tail, is brown
I wonder if brown would have been an issue, even without the animal context, as a search for object with these properties wouldn't have turned up the cent which is copper color
it'd be interesting to ask gp3 what color is a cent and then redo the riddle with that
Re:Teaching GPT-3 to Identify Nonsense
by alexeldeib on Wednesday July 29, 2020 @06:09PM
(#23992137)
A head, tail, is brown, no legs. For an animal that’s got me thinking of fish rather than typos :)