News for hackers. Stuff that matters.
Re:What Are Transformer Models and How Do They Work? by andai on Saturday April 15, 2023 @12:10AM (#35577530)
Here's the original paper: https://arxiv.org/abs/1706.03762
Re:What Are Transformer Models and How Do They Work? by jaidhyani on Sunday April 16, 2023 @12:44PM (#35591625)
The original paper is very good but I would argue it's not well optimized for pedagogy. Among other things, it's targeting a very specific application (translation) and in doing so adopts a more complicated architecture than most cutting-edge modes actually use (encoder-decoder instead of just one or the other). The writers of the paper probably didn't realize they were writing a foundational document at the time. It's good for understanding how certain conventions developed and important historically - but as someone who did read it as an intro to transformers, in retrospect I would have gone with other resources (e.g. "The Illustrated Transformer").