Re:What Are Transformer Models and How Do They Work?
by oergiR on Saturday April 15, 2023 @03:41AM
(#35578518)
I agree except for (6). A language model assigns probabilities to sequences. The model needs normalised distributions, eg using a softmax, so that’s the right way of thinking about it.
[ Parent
| on HN ]
Re:What Are Transformer Models and How Do They Work?
by jaidhyani on Sunday April 16, 2023 @01:05PM
(#35591867)
This is true in general but not in the use case they presented. If they had explained why a normalized distribution is useful it would have made sense - but they just describe this as pick-the-top-answer next-word predictor, which makes the softmax superfluous.