News for hackers. Stuff that matters.
Re:Revealing example of self-attention, the building block o... by pizza on Sunday April 30, 2023 @01:49AM (#35759978)
Here’s my one-liner explanation: attention is the sum of a dict of functions evaluated on input data. But the data is matrices, the dict keys are matrices, and the functions that are found in the dict are also matrices.

There are no comments on this comment yet.