Re:Revealing example of self-attention, the building block o...
by pizza on Sunday April 30, 2023 @01:49AM
(#35759978)
Here’s my one-liner explanation: attention is the sum of a dict of functions evaluated on input data. But the data is matrices, the dict keys are matrices, and the functions that are found in the dict are also matrices.
[ Parent
| on HN ]
There are no comments on this comment yet.