News for hackers. Stuff that matters.

A comment on Transformer learning explained: Coinductive guide to inductive transformer heads. Parent comment

Re:Transformer learning explained: Coinductive guide to indu... by adamnemecek on Tuesday February 28, 2023 @08:03PM (#34977358)
Just FYI when you lookup visualizations of the attention matrices, they tend to be somewhat symmetric.
[ Parent | on HN ]

There are no comments on this comment yet.