News for hackers. Stuff that matters.

A comment on Formal Algorithms for Transformers.

Re:Formal Algorithms for Transformers by mrhether on Thursday July 21, 2022 @12:38PM (#32181487)
familiar with basic ML terminology might be an understatement
[ Parent | on HN ]
Re:Formal Algorithms for Transformers by godelski on Thursday July 21, 2022 @05:19PM (#32185230)
If you're starting from scratch scratch, these might be of more use to you. Second focuses on vision transformers, but all the concepts still apply.

https://jalammar.github.io/illustrated-transformer/

https://medium.com/pytorch/training-compact-transformers-fro...

[ Parent | on HN ]
Re:Formal Algorithms for Transformers by uoaei on Thursday July 21, 2022 @12:40PM (#32181509)
This field moves very quickly. I don't think anyone can be expected to keep up who don't make it a weekly study subject or are actively employed in it.
[ Parent | on HN ]