gman_
Re:Llama2.c: inference llama 2 in one file of pure C
on Monday July 24, 2023 @04:53AM (#36844791)
Wow! This is the most fun I've had programming in a while. Thanks for sharing
Re:Llama2.c: inference llama 2 in one file of pure C
on Monday July 24, 2023 @04:45AM (#36844718)
Looks like you spoke too soon, I'm clocking 340+ tokens per second with my improved Rust implementation, compared to 106 with the original C. That being said, I didn't share this for any reason other than to share ideas and promote learning. Cheers
Re:Llama2.c: inference llama 2 in one file of pure C
on Sunday July 23, 2023 @11:50PM (#36842940)
Here's a Rust version in case anyone's curious what it would look like. It also clocks 106 tokens/second in release mode.
https://github.com/garrisonhess/llama2.c/blob/517a1a3e487f31...