News for hackers. Stuff that matters.
Re:OpenLLaMA: An Open Reproduction of LLaMA by zoogeny on Wednesday May 03, 2023 @11:10AM (#35803327)
Andrej Karpathy's Zero to Hero video series [1] is a good middle ground. It isn't super low-level but it also isn't super high-level. I think seeing how the pieces actually fit together in a working project is valuable to get a real understanding.

After going through this series I can say I basically understand weights, tokens, back-propagation, layers, embeddings, etc.

1. https://karpathy.ai/zero-to-hero.html

Re:OpenLLaMA: An Open Reproduction of LLaMA by data_maan on Wednesday May 03, 2023 @12:28PM (#35804341)
When was this published? Is this an older tutorial by Karpathy?

Just curious, didn't see any date...

Re:OpenLLaMA: An Open Reproduction of LLaMA by knutzui on Wednesday May 03, 2023 @01:11PM (#35804934)
The first class is 8 months old and the latest one is 3 months old. If you click on the links, they'll direct you to YouTube videos.
Re:OpenLLaMA: An Open Reproduction of LLaMA by rini17 on Wednesday May 03, 2023 @01:12PM (#35804948)
On youtube you can. First video 8 months ago.
Re:OpenLLaMA: An Open Reproduction of LLaMA by CamperBob2 on Wednesday May 03, 2023 @01:45PM (#35805391)
I'm working my way through that series now. He really is a good teacher -- I keep waiting for the inevitable "Next, draw the rest of the fucking owl" moment, but so far he does seem to be sticking to his commitment to a from-scratch approach.