Re:Let's Think Dot by Dot: Hidden Computation in Transformer...
by PoignardAzur on Sunday April 28, 2024 @05:26AM
(#40187241)
Basically in a chain of N tokens, the state of the token at layer L reflects L * N / 2 intermediary states worth of info. (In practice a lot less, since attention is a somewhat noisy channel between tokens.)
[ Parent
| on HN ]
There are no comments on this comment yet.