Re:Teaching GPT-3 to Identify Nonsense
by red75prime on Thursday July 30, 2020 @11:41PM
(#24005426)
GPT-3 generates output word by word. And it doesn't make distinctions between prompt words and words it has generated. Generated words are selected somewhat randomly (if temperature >0). So "the train of GPT-3 thought" can be easily derailed by unfortunate random selection of a single word.
When it generated "I don't know what you mean by", it had to continue with something relevant. In this case "no legs" was selected.
In other words GPT-3 doesn't have machinery to stop and think "What I'm rambling about?".
When it generated "I don't know what you mean by", it had to continue with something relevant. In this case "no legs" was selected.
In other words GPT-3 doesn't have machinery to stop and think "What I'm rambling about?".