News for hackers. Stuff that matters.
Re:How GPT3 Works – Visualizations and Animations by ypcx on Monday July 27, 2020 @10:37PM (#23971763)
Yes this seems like an early work in progress, compared to Jay's previous Transformer articles.

In addition to your link, I've found a really good Transformer explanation here (backed by a Github repo w/ lively Issues talk): http://www.peterbloem.nl/blog/transformers

Additionally, there's a paper on visualizing self-attention: https://arxiv.org/pdf/1904.02679.pdf

Re:How GPT3 Works – Visualizations and Animations by ypcx on Tuesday July 28, 2020 @08:33AM (#23975022)
Can't edit the post anymore so adding it here - further research reading on improving the current attention model: https://www.reddit.com/r/MachineLearning/comments/hxvts0/d_b...
Re:How GPT3 Works – Visualizations and Animations by m3at on Monday July 27, 2020 @11:59PM (#23972146)
That's a good complement, thank you for the links