News for hackers. Stuff that matters.

A comment on Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper. Parent comment

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by moultano on Wednesday May 17, 2023 @09:41PM (#35983315)
differentiable relative to the model parameters. The attention mechanism does store(key(x), value(x)), followed by lookup(query(y)), where key(), value(), lookup(), query() are all designed to be differentiable.
[ Parent | on HN ]

There are no comments on this comment yet.