News for hackers. Stuff that matters.

A comment on Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20. Parent comment

Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by sturza on Tuesday May 28, 2024 @01:07PM (#40502913)
Do you think grokking leads to proper generalized reasoning? https://arxiv.org/abs/2405.15071
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by bilsbie on Tuesday May 28, 2024 @02:04PM (#40503609)
Any tips on understanding grokking? I’m not following that paper.
[ Parent | on HN ]
Re:Reproduce GPT-2 (124M) in llm.c in 90 minutes for $20 by sturza on Tuesday May 28, 2024 @02:24PM (#40503823)
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets. Overfitting and being cool about it and some new behavior might emerge.
[ Parent | on HN ]