Llm.c – LLM training in simple, pure C/CUDA

Re:Llm.c – LLM training in simple, pure C/CUDA by blackeyeblitzar on Monday April 08, 2024 @05:20PM (#39973850)

It would be great if someone created a tutorial around this explaining exactly how it works and how to do a test training run. I’m aware it’s not feasible to train a “real” model on personal hardware but it would be nice to have a practical learning experience. I’m not sure if there are good alternatives for that.

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by karpathy on Monday April 08, 2024 @07:56PM (#39974942)

I wrote this, which might be a bit helpful: https://github.com/karpathy/llm.c/blob/master/doc/layernorm/...

But if you don't have the background, I'd recommend my YouTube videos, see the Zero To Hero playlist: https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by blackeyeblitzar on Monday April 08, 2024 @08:05PM (#39975009)

Thank you so much for responding. I will definitely check these out and also pass it on to others who might be interested.

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by MAMAMassakali on Tuesday April 09, 2024 @04:22AM (#39977220)

Thank you so much for the Zero To Hero playlist!

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by vineyardmike on Monday April 08, 2024 @07:40PM (#39974828)

The author has a whole series where he does exactly that. YouTube videos, code examples, documentation, everything. Explains the math, explains how to code it, explains the architecture. Everything.

[ Parent | on HN ]