News for hackers. Stuff that matters.
Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by rafi25 on Wednesday May 17, 2023 @03:42PM (#35980244)
ELI5: In neural networks we could take data numbers, multiply by a constant, add a constant, add them to each other, and then do the same to the results again and again. And you could learn the right constants using labeled data. But we couldn’t multiply data numbers against each other. So in transformers you can. But you must multiply all numbers against all numbers in a specific sample, so it’s heavy.

There are no comments on this comment yet.