Re:SmolGPT: A minimal PyTorch implementation for training a ...
by c0wb0yc0d3r on Thursday January 30, 2025 @12:31AM
(#42875185)
I don’t have a solid ML foundation, and it’s been a decade or more since I’ve worked with linear algebra.
For now I think that might be too deep for what I’m after. I’m at the beach looking out at the vast ocean that is machine learning and LLMs.
If not would invest the time in a decent course, there are plenty online, even offline if you are close enough to where its offered. I took one from Andrew Ng on Coursera years ago, which used Matlab. There are probably much better, more up-to-date options now, especially now that LLMs are very in-vogue. The fundamentals such as gradient descent, ANNs and back-propagation however, is still relevant, and hasn't changed much.
Trying to understand what code is doing without that foundation will be an exercise in futility.