News for hackers. Stuff that matters.
Re:Transformer Attention is off by one by nborwankar on Monday July 24, 2023 @02:49PM (#36852620)
Shouldn’t this be called Regularized SoftMax? Adding 1 in the denominator looks a lot like a regularization in other ML contexts.

There are no comments on this comment yet.