News for hackers. Stuff that matters.

A comment on Llama2.c: inference llama 2 in one file of pure C. Parent comment

Re:Llama2.c: inference llama 2 in one file of pure C by gman_ on Monday July 24, 2023 @04:45AM (#36844718)
Looks like you spoke too soon, I'm clocking 340+ tokens per second with my improved Rust implementation, compared to 106 with the original C. That being said, I didn't share this for any reason other than to share ideas and promote learning. Cheers
[ Parent | on HN ]
Re:Llama2.c: inference llama 2 in one file of pure C by steeve on Monday July 24, 2023 @04:51AM (#36844776)
540 tok/s on the C version using -ffast-math and -Ofast

https://twitter.com/karpathy/status/1683301419716313089?s=20

[ Parent | on HN ]
Re:Llama2.c: inference llama 2 in one file of pure C by gman_ on Monday July 24, 2023 @04:53AM (#36844791)
Wow! This is the most fun I've had programming in a while. Thanks for sharing
[ Parent | on HN ]