News for hackers. Stuff that matters.
Re:C++ GPT-2 inference engine by version_five on Saturday July 01, 2023 @07:13PM (#36556310)
Context: https://twitter.com/a1k0n/status/1674644631156555780

"I reimplemented GPT-2 from scratch in C++ as an exercise to really understand the nuts and bolts of LLMs. GPT2-117M isn't a super great model, but it's extremely satisfying to get it to generate basically the same thing as other reference implementations."

"I" refers to the guy that wrote this, I, version_five have nothing to do with it, I just thought it looked cool.

Re:C++ GPT-2 inference engine by metiscus on Saturday July 01, 2023 @09:49PM (#36557368)
Indeed it does, thanks for posting this.
Re:C++ GPT-2 inference engine by eugenhotaj on Saturday July 01, 2023 @10:33PM (#36557615)
This is pretty cool. I had the same idea but in zig: https://github.com/EugenHotaj/zig_gpt2

Not fully finished yet, haven't gotten around to implementing bpe encoding/decoding and only some ops use BLAS.

Re:C++ GPT-2 inference engine by eclectic29 on Sunday July 02, 2023 @12:14AM (#36558161)
I found this extremely helpful even though it's not in C++: https://jaykmody.com/blog/gpt-from-scratch/.
Re:C++ GPT-2 inference engine by JPLeRouzic on Sunday July 02, 2023 @05:39AM (#36559884)
Yes it's very nice from the author to provide this very readable text. I saved this text to read it later.
Re:C++ GPT-2 inference engine by kken on Sunday July 02, 2023 @02:50AM (#36558961)
From the same twitter thread, a dense implementation in less than 100 lines of plan C!

https://github.com/davidar/eigenGPT/tree/c

Re:C++ GPT-2 inference engine by rhelz on Tuesday July 04, 2023 @12:19PM (#36588736)
Well, I downloaded and compiled it (cool! Thanks!) but no matter what prompt I give it, it just prints out gibberish....where do I go now to learn how to properly use it?