News for hackers. Stuff that matters.

A comment on Revealing example of self-attention, the building block of transformer AI models.

Re:Revealing example of self-attention, the building block o... by ybu on Saturday April 29, 2023 @06:44PM (#35757997)
In this context worth calling out Andrej Karpathy's youtube playlist on neural networks [0].

In the last video ("Let's build GPT: from scratch...") Andrej codes up a transformer model, in conjunction with the paper [1].

[0] https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThs...

[1] https://arxiv.org/abs/1706.03762

[ Parent | on HN ]

There are no comments on this comment yet.