News for hackers. Stuff that matters.
Re:Implementing a ChatGPT-like LLM from scratch, step by step by natrys on Saturday January 27, 2024 @01:42PM (#39158453)
Neural Networks: Zero to Hero[1] by Andrej Karpathy

[1] https://karpathy.ai/zero-to-hero.html

Re:Implementing a ChatGPT-like LLM from scratch, step by step by villedespommes on Saturday January 27, 2024 @02:51PM (#39159144)
+1, Andrey is an amazing educator! I'd also recommend his https://youtu.be/kCc8FmEb1nY?si=mP0cQlQ4rcceL2uP and checking out his github repos. MinGPT, for example, implements a small gpt model that's compatible with HF API, whereas more modern nanoGPT shows how to use newer features such as flash attention. The quality of every video, every blog post is just so high.