News for hackers. Stuff that matters.

A comment on We Found an Neuron in GPT-2. Parent comment

Re:We Found an Neuron in GPT-2 by minkzilla on Thursday February 16, 2023 @10:39PM (#34830432)
Your friend is thinking ahead that the next word would be apple When they say ‘an’.

But GPT can’t think ahead what token it will add after the one it is on. Or can it? It could “predict” internally the word apple for the next “meaningful” word and output ‘an’ because of this.

[ Parent | on HN ]
Re:We Found an Neuron in GPT-2 by IgorPartola on Thursday February 16, 2023 @11:46PM (#34830897)
Or it could keep outputting articles where appropriate until it outputs an “an” and then output apple having teed itself up.
[ Parent | on HN ]