This was a very enjoyable read, thank you! You do a great job in making these concepts understandable.

The self attention mechanisms caught my eyes. Going to look into implementing something like that for a toy dataset. Thanks for the inspiration