News for hackers. Stuff that matters.
Re:Transformer Attention is off by one by janalsncm on Monday July 24, 2023 @04:11PM (#36853572)
I wouldn’t quite say its value is zero. It’s worth something, but a lot less than if it had been shown to work empirically.

Explainers and their folksy, imprecise tone are good for things we already know are true. I’m skeptical on things which are unproven.

Re:Transformer Attention is off by one by taberiand on Monday July 24, 2023 @05:02PM (#36854147)
Perhaps what it's worth depends on if you've applied the Softmax or the Softmax1 to the value of the blogpost?