News for hackers. Stuff that matters.
Re:Teaching GPT-3 to Identify Nonsense by red75prime on Thursday July 30, 2020 @11:41PM (#24005426)
GPT-3 generates output word by word. And it doesn't make distinctions between prompt words and words it has generated. Generated words are selected somewhat randomly (if temperature >0). So "the train of GPT-3 thought" can be easily derailed by unfortunate random selection of a single word.

When it generated "I don't know what you mean by", it had to continue with something relevant. In this case "no legs" was selected.

In other words GPT-3 doesn't have machinery to stop and think "What I'm rambling about?".

There are no comments on this comment yet.