News for hackers. Stuff that matters.
Re:Revealing example of self-attention, the building block o... by btown on Saturday April 29, 2023 @08:29PM (#35758634)
This MIT lecture on recurrent neural networks provides a lot of context on why attention was so groundbreaking, and how things were solved before: https://www.youtube.com/watch?v=ySEx_Bqxvvo

And for the Attention Is All You Need paper itself, Andrej Karpathy's "GPT From Scratch" talk is a really great walkthrough of how to make this work in practice, all condensed into two hours of live coding: https://www.youtube.com/watch?v=kCc8FmEb1nY

Re:Revealing example of self-attention, the building block o... by hn_throwaway_99 on Saturday April 29, 2023 @09:20PM (#35758873)
That whole MIT course, Intro to Deep Learning, is great and a really nice introduction to the field.