Re:Transformer Attention is off by one
by nborwankar on Monday July 24, 2023 @02:49PM
(#36852620)
Shouldn’t this be called Regularized SoftMax? Adding 1 in the denominator looks a lot like a regularization in other ML contexts.
[ Parent
| on HN ]
There are no comments on this comment yet.