The Transformer Family

Re:The Transformer Family by heyitsguay on Sunday January 29, 2023 @02:55PM (#34572007)

For a while now, an answer I've seen is to start with "Attention Is All You Need", the original Transformers paper. It's still pretty good, but over the past year I've led a few working sessions on grokking transformer computational fundamentals and they've turned up some helpful later additions that simplify and clarify what's going on.

You can quickly get overwhelmed by the million good resources out there so I'll keep it to these three. If you have a strong CS background, they'll take you a long way:

(1) Transformers from Scratch: https://peterbloem.nl/blog/transformers

(2) Attention Is All You Need: https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547de...

(3) Formal Algorithms for Transformers: https://arxiv.org/abs/2207.09238

[ Parent | on HN ]

Re:The Transformer Family by bfung on Sunday January 29, 2023 @03:37PM (#34572457)

* * *

[ Parent | on HN ]

Re:The Transformer Family by peterfirefly on Sunday January 29, 2023 @05:00PM (#34573241)

Yannic Kilcher is great but this video worked better for me:

"LSTM is dead. Long live transformers!" (Leo Dirac): https://www.youtube.com/watch?v=S27pHKBEp30

[ Parent | on HN ]

Re:The Transformer Family by nerdponx on Sunday January 29, 2023 @03:35PM (#34572439)

* * *

[ Parent | on HN ]

Re:The Transformer Family by peterfirefly on Sunday January 29, 2023 @05:01PM (#34573249)

That problem doesn't really disappear with teachers and classes ;)

[ Parent | on HN ]

Re:The Transformer Family by d4rkp4ttern on Monday January 30, 2023 @08:04AM (#34579069)

I second the recommendation for Peter Bloem’s tutorial.

I’m also about to read the transformer chapter from this excellent upcoming book by Simon Prince:

udlbook https://udlbook.github.io/udlbook/

[ Parent | on HN ]