Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by hoosieree on Wednesday May 17, 2023 @08:49PM (#35982996)

Thanks for this explanation. I couldn't wrap my mind around the "differentiable hash table" analogy, but "distribution of keys" -> "distribution of values" starts to click.

I'm not an ML expert either but I have taken graduate level courses and published papers with "machine learning" in the title, so I feel like I should be able to understand these things better. The field just moves so fast. It's a lot of work to keep up. Easy-to-digest explanations like this are underrated.

[ Parent | on HN ]

Re:Ask HN: Can someone ELI5 Transformers and the “Attention ... by HybridCurve on Thursday May 18, 2023 @05:05PM (#35994042)

>The field just moves so fast. It's a lot of work to keep up. Easy-to-digest explanations like this are underrated.

This is really the truth. I can't possibly understand how people in this field who are talented can still keep up. I have a binder full of seminal papers that I have to cull to make room for more recent and relevant research every few months. I feel there is a lot of potential in simplifying the details of the mechanisms that drive a lot of it, but nobody has time to stop, consolidate the information and publish it. And if they did, it would just be another outdated textbook in a few years.

[ Parent | on HN ]