Ask HN: How to Learn LLM Fundamentals

Posted by AETackaberry on Thursday March 30, 2023 @12:54AM
from the ask-hn dept.

As a Software Engineer that is interested in learning LLM fundamentals, what is your recommended course of action? Are there any courses or curricula you would recommend?

( 10 comments | 4 points | on HN )

Re:Ask HN: How to Learn LLM Fundamentals by hislaziness on Thursday March 30, 2023 @01:06AM (#35368267)

I really liked this series. https://karpathy.ai/zero-to-hero.html

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by AETackaberry on Thursday March 30, 2023 @01:12AM (#35368327)

I am definitely planning to start with this series, seems like a no-brainer!

The part I'm unsure of is what next? Seems like diving into papers might be a decent leap. Who knows, maybe GPT-4 can help me bridge the gap

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by hislaziness on Thursday March 30, 2023 @02:07AM (#35368695)

Papers would be good next step. The concept and the algorithms do not seem very complex. The training and fine tuning is what needs effort and infrastructure.

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by quickthrower2 on Thursday March 30, 2023 @01:28AM (#35368440)

Thanks that series looks quite fun to do

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by seydor on Thursday March 30, 2023 @06:05AM (#35370365)

There s this guy who has made in depth videos about transformers https://www.youtube.com/watch?v=O3xbVmpdJwU

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by is_true on Thursday March 30, 2023 @08:08AM (#35371365)

I think this blog posts is nice overview to help you go deeper in the topics mentioned: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by jytechdevops on Thursday March 30, 2023 @01:33PM (#35376314)

What a coincidence, I actually ran across this github repo for LLM fundamentals today: https://gist.github.com/rain-1/eebd5e5eb2784feecf450324e3341...

gl on your journey

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by TroyZ on Thursday March 30, 2023 @10:24PM (#35381672)

For a high level overview of transformers/GPT, you can check out this article: https://medium.com/design-bootcamp/how-chatgpt-really-works-...

The article provides references to the original papers and other articles explaining the subject, which can be great sources to dive deeper.

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by felipelalli on Friday March 31, 2023 @01:48AM (#35383108)

http://www.iasylum.net/writings/2023-03-29-if-you-are-curiou...

[ Parent | on HN ]

Re:Ask HN: How to Learn LLM Fundamentals by bfg_damien on Sunday April 02, 2023 @05:58PM (#35415703)

Here are a few resources that might help get you started:

1. A thread explaining the internal working of transformers: https://twitter.com/hippopedoid/status/1641432291149848576?s...

2. Paper by DeepMind which provides pseudo-code for important algorithms for Transformer models: https://arxiv.org/pdf/2207.09238.pdf

3. Another thread specifically on large language models: https://twitter.com/cwolferesearch/status/164044611134855577...

Once again these are not courses per se, but do provide intuitive explanations for how transformers work. There is also the nanoGPT series of videos by Karpathy on youtube. First video here: https://www.youtube.com/watch?v=kCc8FmEb1nY

[ Parent | on HN ]