News for hackers. Stuff that matters.

A comment on Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper. Parent comment

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by nafey on Thursday May 18, 2023 @12:32AM (#35984150)
When one says "attention is all you need" the implication is that some believe that you need something more than just attention. What is that something which has been demonstrated as unneeded? Is it a theory of how language works?
[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by kylewatson on Thursday May 18, 2023 @01:05AM (#35984291)
I think this will help: https://medium.datadriveninvestor.com/attention-in-rnns-321f...
[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by trenchgun on Thursday May 18, 2023 @02:10AM (#35984590)
Recursion. Before transformers attention was used in recurrent neural networks. "attention is all you need" showed that you can just drop the recursion and just use attention, and the outcome is that you get a very nicely parallelizable architechture, allowing more efficient training.
[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by Sammi on Monday May 22, 2023 @06:55AM (#36029787)
Finally someone cut to chase. Thanks you!
[ Parent | on HN ]