Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart
by evolvingstuff on Monday April 15, 2024 @01:40PM
(#40043477)
You are correct, that is an error in an otherwise great video. The k+1 token is not merely a function of the kth vector, but rather all prior vectors (combined using attention). There is nothing "special" about the kth vector.
[ Parent
| on HN ]
There are no comments on this comment yet.