News for hackers. Stuff that matters.
Re:Implementing a ChatGPT-like LLM from scratch, step by step by malermeister on Saturday January 27, 2024 @03:18PM (#39159414)
Thank you for the answer! What is the knowledge that your book requires? If I have a lot of software dev experience and sorta kinda remember algebra from uni, would it be a good fit?
Re:Implementing a ChatGPT-like LLM from scratch, step by step by rasbt on Saturday January 27, 2024 @08:12PM (#39161550)
Good question. I think a Python background is strongly recommended. PyTorch knowledge would be a nice to have (although I've written a comprehensive 40 page intro for the Appendix, which is also already available). From a math perspective, I think it should be gentle. I'm introducing dot products in Chapter 3, but I also explain how you could do the same with for-loops. Same with matrix multiplication. I'm bad at estimating requirements, but I hope this should be sufficient.