News for hackers. Stuff that matters.

A comment on Transformer Attention is off by one. Parent comment

Re:Transformer Attention is off by one by bearzoo on Monday July 24, 2023 @03:04PM (#36852833)
think the author is talking about having to fix the extra vector in V to be zeros and making sure to not compute/apply gradients to it
[ Parent | on HN ]

There are no comments on this comment yet.