Re:Understanding Transformers Using a Minimal Example
by CGMthrowaway on Wednesday September 03, 2025 @08:26PM
(#45121968)
I've seen it but I don't believe I've watched it all the way through. I will now
Re:Understanding Transformers Using a Minimal Example
by imtringued on Thursday September 04, 2025 @06:47AM
(#45125756)
I personally would rather recommend people to just look at these architectural diagrams [0] and try to understand them. There is the caveat that they do not show how attention works. For that you need to understand softmax(QK^T)V and multi head attention being a repetition of this multiple times. GQA, MHA, etc just messes around with reusing Q or K or V in clever ways.
Re:Understanding Transformers Using a Minimal Example
by rhdunn on Thursday September 04, 2025 @07:18AM
(#45125969)
There's also various videos by Welch Labs that are very good. -- https://www.youtube.com/@WelchLabsVideo/videos
https://youtu.be/wjZofJX0v4M