This lecture by Leo Dirac was extremely helpful to finish off with, not only because it actually includes some pseudocode but it also revisits some key topics and covers why transformers are needed.
One of the big confusion points for me was that the concept of ATTENTION and SELF-ATTENTION are not the same thing.
1. Chapters 7, 9, 10
https://web.stanford.edu/~jurafsky/slp3/
This was really useful to really build up to the concepts of attention (although the actual attention section is still brief).
2. https://jalammar.github.io/visualizing-neural-machine-transl...
This was great for visualization and understanding that attention wasn't exclusively for transformers and actually was for RNNs first.
3. https://jalammar.github.io/illustrated-transformer/
Getting to understand how transformers actually work visually.
4. https://www.youtube.com/watch?v=S27pHKBEp30
This lecture by Leo Dirac was extremely helpful to finish off with, not only because it actually includes some pseudocode but it also revisits some key topics and covers why transformers are needed.
One of the big confusion points for me was that the concept of ATTENTION and SELF-ATTENTION are not the same thing.
Hope this helps.