News for hackers. Stuff that matters.
Re:Transformer Attention is off by one by llwu on Monday July 24, 2023 @08:28PM (#36856134)
This is a cool idea - it could be tested it by seeing whether W_V tends to map <bot> to 0, and whether <bot> tends to get attended to.

There are interesting things to said whether or not this turns out to be the case.

There are no comments on this comment yet.