News for hackers. Stuff that matters.
Re:We Found an Neuron in GPT-2 by tqi on Friday February 17, 2023 @04:12PM (#34840647)
Thanks for responding, and for the cool article! Agree this is pretty tangential to the rest of the article.

So (again, I am very much not an expert so please correct me if I'm wrong) I guess my analogy would be if instead of predicting one word at a time, it predicted one letter at a time. At some point, would only be one word that could fit. So if prompted with "SENTE", and it returned "N", that doesn't mean that it's thinking ahead to the "CE" / knows that it is spelling "SENTENCE" already.

Is that a correct way to think of it?

There are no comments on this comment yet.