News for hackers. Stuff that matters.

A comment on Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper. Parent comment

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by throwawaymaths on Wednesday May 17, 2023 @05:42PM (#35981635)
That's exactly the point, though! It's surprising. A hashtable is a map from keys to values. Making it differentiable means that a small change in the key also makes a small change in the value!
[ Parent | on HN ]

There are no comments on this comment yet.