Llama2.c: inference llama 2 in one file of pure C

Re:Llama2.c: inference llama 2 in one file of pure C by wtarreau on Monday July 24, 2023 @03:21AM (#36844138)

As often with Rust, someone transliterates something that already exists just because they can, without providing any benefit at all. Sometimes it even results in fragmenting the community efforts to improve the project.

[ Parent | on HN ]

Re:Llama2.c: inference llama 2 in one file of pure C by voz_ on Monday July 24, 2023 @04:31AM (#36844609)

Can you chill? Stuff like this is super useful. The original c file is educational, so is this. And now by having it two ways, we have a tiny little Rosetta Stone for folks that wanna learn.

[ Parent | on HN ]

Re:Llama2.c: inference llama 2 in one file of pure C by jheuel on Monday July 24, 2023 @07:52AM (#36846383)

Might also help as input to the next iteration of LLMs

[ Parent | on HN ]

Re:Llama2.c: inference llama 2 in one file of pure C by gman_ on Monday July 24, 2023 @04:45AM (#36844718)

Looks like you spoke too soon, I'm clocking 340+ tokens per second with my improved Rust implementation, compared to 106 with the original C. That being said, I didn't share this for any reason other than to share ideas and promote learning. Cheers

[ Parent | on HN ]

Re:Llama2.c: inference llama 2 in one file of pure C by steeve on Monday July 24, 2023 @04:51AM (#36844776)

540 tok/s on the C version using -ffast-math and -Ofast

https://twitter.com/karpathy/status/1683301419716313089?s=20

[ Parent | on HN ]

Re:Llama2.c: inference llama 2 in one file of pure C by gman_ on Monday July 24, 2023 @04:53AM (#36844791)

Wow! This is the most fun I've had programming in a while. Thanks for sharing

[ Parent | on HN ]

Re:Llama2.c: inference llama 2 in one file of pure C by elteto on Monday July 24, 2023 @11:38AM (#36849716)

You should list your email in your HN profile. That way the Internet could check with you to see if you approve whenever someone starts a new personal project.

[ Parent | on HN ]