Micrograd-CUDA: adapting Karpathy's tiny autodiff engine for GPU acceleration

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by cataPhil on Wednesday March 20, 2024 @01:19PM (#39769504)

I recently decided to turbo-teach myself basic cuda with a proper project. I really enjoyed Karpathy’s micrograd (https://github.com/karpathy/micrograd), so I extended it with cuda kernels and 2D tensor logic. It’s a bit longer than the original project, but it’s still very readable for anyone wanting to quickly learn about gpu acceleration in practice.

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by quibono on Wednesday March 20, 2024 @05:47PM (#39772385)

Is your work publicly available anywhere? I'd love to have a look if possible.

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by cataPhil on Wednesday March 20, 2024 @05:52PM (#39772435)

Hi :) Yes it's in the post url or here: https://github.com/mlecauchois/micrograd-cuda

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by quibono on Wednesday March 20, 2024 @06:26PM (#39772756)

Oh sorry, I didn't realise you were OP :) Thanks!

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by lagrange77 on Wednesday March 20, 2024 @06:03PM (#39772538)

Awesome, thank you for publishing it. I was planning the same, but didn't find the time yet.

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by coolThingsFirst on Wednesday March 20, 2024 @10:55PM (#39774509)

Lovely project, what did you find as the beat place to learn about cuda?

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by cataPhil on Thursday March 21, 2024 @04:19AM (#39775966)

Thanks! I highly recommend the book Programming Massively Parallel Processors

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by st380752143 on Thursday March 21, 2024 @07:14PM (#39785577)

This is the second time this week that people have recommended this boot while I was searching for information about CUDA. Thanks!

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by lairv on Thursday March 21, 2024 @03:29AM (#39775684)

If you ever feel like improving your matmul kernel you can check this blog https://siboehm.com/articles/22/CUDA-MMM

[ Parent | on HN ]

Re:Micrograd-CUDA: adapting Karpathy's tiny autodiff engine ... by cataPhil on Thursday March 21, 2024 @04:20AM (#39775972)

Thanks! I will probably do that this weekend

[ Parent | on HN ]