News for hackers. Stuff that matters.

A comment on Implementing a ChatGPT-like LLM from scratch, step by step. Parent comment

Re:Implementing a ChatGPT-like LLM from scratch, step by step by larme on Saturday January 27, 2024 @01:40PM (#39158425)
https://jaykmody.com/blog/gpt-from-scratch/ for a gpt2 inference engine in numpy

then

https://www.dipkumar.dev/becoming-the-unbeatable/posts/gpt-k... for adding a kv cache implementation

[ Parent | on HN ]
Re:Implementing a ChatGPT-like LLM from scratch, step by step by larme on Saturday January 27, 2024 @01:48PM (#39158503)
I'd like to add that most of these text only talking about inference part. This book (I also purchased the draft version) has training and finetuning in the TOC. I assume it will include materials about how to do training and finetuning from scratch.
[ Parent | on HN ]