Re:The Illustrated Transformer
by krat0sprakhar on Monday December 22, 2025 @03:20PM
(#46358521)
Do you have a tutorial that I can follow?
[ Parent
| on HN ]
Re:The Illustrated Transformer
by roadside_picnic on Monday December 22, 2025 @04:27PM
(#46359303)
The most valuable tutorial will be translating from the paper itself. The more hand holding you have in the process, the less you'll be learning conceptually. The pure manipulation of matrices is rather boring and uninformative without some context.
I also think the implementation is more helpful for understanding the engineering work to run these models that getting a deeper mathematical understanding of what the model is doing.
If you have 20 hours to spare I highly recommend this youtube playlist from Andrej Karpathy
https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...
It starts with the fundamentals of how backpropagation works then advances to building a few simple models and ends with building a GPT-2 clone. It won't taech you everything about AI models but it gives you a solid foundation for branching out.