News for hackers. Stuff that matters.
Re:3Blue1Brown: Visualizing Attention, a Transformer's Heart by evolvingstuff on Monday April 15, 2024 @01:40PM (#40043477)
You are correct, that is an error in an otherwise great video. The k+1 token is not merely a function of the kth vector, but rather all prior vectors (combined using attention). There is nothing "special" about the kth vector.

There are no comments on this comment yet.