News for hackers. Stuff that matters.
Re:RedPajama v2 Open Dataset with 30T Tokens for Training LLMs by npsomaratna on Monday October 30, 2023 @11:12PM (#38079807)
Yup. I found this article quite enlightening:

https://espadrine.github.io/blog/posts/chinchilla-s-death.ht...

Re:RedPajama v2 Open Dataset with 30T Tokens for Training LLMs by rushingcreek on Tuesday October 31, 2023 @12:36AM (#38080378)
Phenomenal blog post about scaling laws.