News for hackers. Stuff that matters.

A comment on Transformers from Scratch. Parent comment

Re:Transformers from Scratch by jaymody on Wednesday April 26, 2023 @11:34AM (#35715398)
I wrote a minimal implementation in NumPy here (the forward pass code is only 40 lines): https://github.com/jaymody/picoGPT

And also a related blog post: https://news.ycombinator.com/item?id=34726115

Although this is for a decoder-only transformer (aka GPT) and doesnt include the encoder part.

[ Parent | on HN ]
Re:Transformers from Scratch by newhouseb on Wednesday April 26, 2023 @02:22PM (#35717959)
And I adapted Jay's work to Typescript (without the numpy obviously, just raw typescript/javascript): https://github.com/newhouseb/potatogpt
[ Parent | on HN ]