News for hackers. Stuff that matters.

A comment on Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20. Parent comment

Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by lagrange77 on Tuesday May 28, 2024 @01:44PM (#40503358)
Thank you for the effort you put in your educational work, it helped me and others a lot! In fact, i'm training my nanoGPT version right now. :)

> Ultimately my interest in llm.c is to have a nice, clean, minimal, super dependency-light repo in direct C/CUDA implementation, which I find aesthetically pleasing.

Also, it's awesome that you spend your time on your passion.

Any plans on making a video series on llm.c? :D

[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by karpathy on Tuesday May 28, 2024 @01:56PM (#40503512)
Yes definitely. Related tweet of mine:

https://x.com/karpathy/status/1760388761349927356?lang=en

1. Build the thing

2. Build the ramp

Currently on step 1 :). It helps to build it first so you know where you are going, and then you can more easily re-build it when you're vector pointed at the end result.

[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by htrp on Tuesday May 28, 2024 @02:32PM (#40503933)
Everytime you take gardening leave, you build something new and interesting!
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by lagrange77 on Tuesday May 28, 2024 @02:15PM (#40503739)
That's fantastic. My gradient field is pointing towards it.

Thank you again!

[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by LorenzoGood on Tuesday May 28, 2024 @11:03PM (#40508012)
I love when you leave your job.
[ Parent | on HN ]