News for hackers. Stuff that matters.
Re:Teaching ChatGPT to Speak My Son’s Invented Language by fenomas on Tuesday April 11, 2023 @05:18AM (#35522670)
An attention mechanism is one or more layers in the neural network. When someone talks about attention altering the input vectors, they're referring to what those layers are doing and how data is transformed as it passes through them. But zooming out to the big picture, a neural network is a bunch of layers full of weights, and none of the weights changes except during training (including the weights in the attention layers).

There are no comments on this comment yet.