News for hackers. Stuff that matters.

A comment on Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper.

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by jgotti92 on Wednesday May 17, 2023 @12:57PM (#35978108)
Check this explanation from Karpathy which provides really good intuition https://youtu.be/9uw3F6rndnA

Basically tokens “talk” to each other and say this is what i have and this is what i offer.

[ Parent | on HN ]
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by saaaaaam on Wednesday May 17, 2023 @01:58PM (#35978976)
Your summary of this is very good. It’s a really simple and intuitive way to express something very complex.
[ Parent | on HN ]