Re:Llama 3.1 in C
by AMICABoard on Wednesday July 24, 2024 @04:04AM
(#41054652)
Oh thanks bro, nope it uses the simple llama 2 rope with tetha changed to 50k to match llama 3's. I'll check your python PR, have a deeper look at the meta llama 3 & 3.1 implementation and hack together something soonish. Awesome!
[ Parent
| on HN ]
Re:Llama 3.1 in C
by danielhanchen on Wednesday July 24, 2024 @04:07AM
(#41054670)
Np! Oh ye my PR is just a random conversion from Python to C - haven't done C in ages though so probs won't even compile!
Re:Llama 3.1 in C
by AMICABoard on Wednesday July 24, 2024 @04:18AM
(#41054744)
To be honest your pr and these notes are super helpful cos, otherwise I'd have been too lazy to read up the original implementation, but I can't merge it. Will make the fix soonish and credit it to you. I'll tell you a secret, nobody is good at C except for a few wizards like woz, Linus, jart, Knuth, Bellard and the like. I am also a total looser at C. I just love C. And also the base of this is mostly Karpathy's work and other awesome folks who are way better at everything than me.
Re:Llama 3.1 in C
by danielhanchen on Wednesday July 24, 2024 @11:29AM
(#41058074)
Oh thanks! :) Oh ye Karpathy's repo is pretty sick! Well still great work on it! If you need any help on it, feel free to ask!
Re:Llama 3.1 in C
by AMICABoard on Wednesday July 24, 2024 @12:36PM
(#41058825)
Will do! Thanks :)
Re:Llama 3.1 in C
by danielhanchen on Thursday July 25, 2024 @03:44AM
(#41065894)
:)