Re:Chinchilla Scaling: A Replication Attempt
by warbaker on Thursday April 18, 2024 @03:27PM
(#40079851)
Calling this a "replication attempt" implied to me that they tried to replicate the Chinchilla Scaling paper and found that it did not replicate, which would be a very big deal!
Instead, they just redid the analysis based on a figure in the paper and found that the old model with slightly different parameters gave a better fit to the data. This is a valuable contribution, but a bit over-stated by the paper title, and the confrontational, "gotcha" tone of the paper is unwarranted.
A better framing would have been something like "Chinchilla Scaling: Reanalyzed".
Re:Chinchilla Scaling: A Replication Attempt
by ege_erdil on Thursday April 18, 2024 @05:33PM
(#40080948)
one of their three approaches does not replicate and it's because of a software bug in the optimizer they used, i don't know what else we were supposed to say
Instead, they just redid the analysis based on a figure in the paper and found that the old model with slightly different parameters gave a better fit to the data. This is a valuable contribution, but a bit over-stated by the paper title, and the confrontational, "gotcha" tone of the paper is unwarranted.
A better framing would have been something like "Chinchilla Scaling: Reanalyzed".