Re:Llama from Scratch (or how to implement a paper without c...
by earth2mars on Thursday August 10, 2023 @03:15AM
(#37073002)
Use perplexity.ai . Why not use AI to learn AI! The good thing I like about this tool is that it gives citations, so that you can learn further beyond summarization it does.
Re:Llama from Scratch (or how to implement a paper without c...
by mike_hearn on Thursday August 10, 2023 @03:28AM
(#37073088)
I've been using a pretty random mix of things including the PyTorch tutorial, some of the tutorials on how transformers work that got posted here months ago, reading papers, and (of course) asking GPT4.
I would say that learning at this level is probably not so important. What's far more important is to know how to use LLMs as an API or library, which is of course 1% coding because the API is so easy and 99% figuring out what their limits are, how best to integrate them into workflows, how to design textual "protocols" to communicate with the AI and so on. Learning how to train a model from scratch is fun but to get competitive results is too expensive, so pragmatism requires focus on being a user for now.
Any pointers / references / books that you’ve found particularly helpful in your learning journey?
I know about Karpathy’s video series (and accompanying repos). Anything else come to mind? Thanks!