Thanks for sharing.

Seconded. It's a hands-on approach starting with implementing a pytorch-like api from the ground up with manual backprop up to implementing a simple transformer / gpt variant in actual pytorch.

That course is fantastic. Just don't be afraid to pause and rewatch. It took me a long time to get through the first three videos.

Would you mind sharing, what level of programming & mathematical background do I need? I know basic python (read python for data analysis) & currently half way through elements of statistical learning. What else do I need to learn?

> half way through elements of statistical learning

If you're able to make it that far on ESL, mathematical background certainly won't hold you back when learning anything "neural networks" related. Specially not a spelled-out practical intro.

> I know basic python (read python for data analysis)

You may want to get more comfortable with programming in general (outside of the data analysis realm), but you can learn everything you're missing while watching Karpathy's series (and referencing the python docs).

Karpathy is great researcher in the field, but that doesn't always translate into good explanation skills automatically.