News for hackers. Stuff that matters.
Re:StarCoder and StarCoderBase: 15.5B parameter models with ... by og_kalu on Tuesday May 16, 2023 @07:24AM (#35960260)
>You already said "only by training on far more data", which is different than "more parameters" being the only option.

I never said more parameters was the only way to increase performance. I said the training data required to reach any arbitrary performance x reduces with parameter size.

It's literally right there in what I wrote.

>a 50 billion parameter model will far outperform a 5 billion one TRAINED ON THE SAME DATA.

There are no comments on this comment yet.