Neural Networks: Zero to Hero[1] by Andrej Karpathy

[1] https://karpathy.ai/zero-to-hero.html

+1, Andrey is an amazing educator! I'd also recommend his https://youtu.be/kCc8FmEb1nY?si=mP0cQlQ4rcceL2uP and checking out his github repos. MinGPT, for example, implements a small gpt model that's compatible with HF API, whereas more modern nanoGPT shows how to use newer features such as flash attention. The quality of every video, every blog post is just so high.

https://jaykmody.com/blog/gpt-from-scratch/ for a gpt2 inference engine in numpy

then

https://www.dipkumar.dev/becoming-the-unbeatable/posts/gpt-k... for adding a kv cache implementation

I'd like to add that most of these text only talking about inference part. This book (I also purchased the draft version) has training and finetuning in the TOC. I assume it will include materials about how to do training and finetuning from scratch.

I added notes to the Jupyter notebooks, I hope they are also readable as standalone from the repo.

I honestly cannot fathom why anyone working in the AI space would find $50 too much to spend to gain a deeper insight into the subject. Creating educational materials requires an insane amount of work, and I can promise, no matter how successful this book is, if rasbt were do the math on income generated over hours spent creating it wouldn't make sense as an hourly rate.

Plenty of other people have this understanding of these topics, and you know what they chose to do with that knowledge? Keep it to themselves and go work at OpenAI to make far more money keeping that knowledge private.

If you want to live in a world where this knowledge is open, at the very least refrain from publicly complaining about a book that cost roughly the same as a decent dinner.

Yeah, I don't think creating educational materials makes sense from an economical perspective, but it's one of my hobbies that gives me joy for some reason :). Hah, and 'insane amount of work' is probably right -- lots of sacrifices to carve out that necessary time.

Not talking about affordability but about following links thinking that I would find another kind of resource. Beyond this case, this happens all the time with click-baity content. Again, if the link was to Amazon or the editors it will be clear associated with a product while Github is associated with open source content. Not being pedantic, just an observation browsing the web.

I would add that I can find 5k of useful resources for USD 50 each. It is not the individual item but the all the context.

Personally, I am not focused on a specific topic such as LLMs but work on an spectrum of topics more akind an analyst job + broad research skills.

Not to be pedantic, but in this case it's probably 30 usd for print and ebook (there are always coupons on the manning website).

> anyone working in the AI space

I would have expected the main target audience to be people NOT working in the AI space, that don’t have any prior knowledge (“from scratch”), just curious to learn how an LLM works.

I'd go with https://course.fast.ai/

It's much more accessible to regular developers, and doesn't make assumptions about any kind of mathematics background. It's a good starting poing after which other similar resources start to make more sense.