Llm.c – LLM training in simple, pure C/CUDA

Re:Llm.c – LLM training in simple, pure C/CUDA by idkwhatimdoin on Monday April 08, 2024 @05:53PM (#39974127)

If I was starting from scratch, what resources should I start with to build up an understanding of what this code does and how to read it? It's quite dense and my knowledge of LLMs is quite minimal. Are these terse variable names standard in LLM-land?

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by vineyardmike on Monday April 08, 2024 @07:39PM (#39974820)

Terse variables are a C thing.

“What resources would I need” -> you’re literally commenting on a teachers content. Karpathy (the author) has a very informative YouTube channel where he goes step by step through everything. He has a ton of repos and tutorials. Dig a little.

If all else fails… Google it.

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by viraptor on Monday April 08, 2024 @08:16PM (#39975083)

> Terse variables are a C thing.

They're a math / toy code thing. Large C projects have long descriptive names just like other languages.

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by idkwhatimdoin on Tuesday April 09, 2024 @12:52AM (#39976264)

> you’re literally commenting on a teachers content.

How am I supposed to know that?

> Karpathy (the author) has a very informative YouTube channel where he goes step by step through everything.

Or that, without knowing that he's a teacher?

> Terse variables are a C thing.

I didn't realize variables had to be so short in C. Glad I write C++ professionally where they've added support for longer variable names.

> If all else fails… Google it.

There's a lot of LLM garbage out there. I got an answer here in a few minutes pointing to Karpathy's course which seems very high quality.

Be kinder.

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by vineyardmike on Tuesday April 09, 2024 @03:03AM (#39976827)

> How am I supposed to know that?

You’re not supposed to know that. You asked a question, and this is you being told the answer.

It’s very convenient that the author of the post is quite literally the world’s most prolific teacher on this topic. Makes it easy to find Karpathy. You shouldn’t be expected to otherwise know that (or else why ask if you knew).

> I didn't realize variables had to be so short in C. Glad I write C++ professionally where they've added support for longer variable names.

This feels like a joke but old C compilers did have variable length limits. This is part of why C historically had shorter variables than other more modern languages.

Sorry if it came off rude, the internet is hard to communicate over.

https://publications.gbdirect.co.uk/c_book/chapter2/keywords...

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by tayo42 on Monday April 08, 2024 @06:28PM (#39974386)

Check out his zero to hero series. Which builds this with python and later pytorch, then probably his other mini C based projects.

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by satokema on Monday April 08, 2024 @08:41PM (#39975205)

As siblings have said, his video series are quite good. But if you're just looking at this repo only, you probably want to look at the python reference implementation. (The C is designed to exactly replicate its functionality.)

[ Parent | on HN ]

Re:Llm.c – LLM training in simple, pure C/CUDA by ygjb on Wednesday April 10, 2024 @02:40PM (#39994215)

If you are starting from scratch?

https://karpathy.ai/zero-to-hero.html

Do those, then come back to this and it will make more sense.

[ Parent | on HN ]