News for hackers. Stuff that matters.
Re:DeepMind’s New Language Model, Chinchilla by eigenvalue on Monday April 11, 2022 @05:33PM (#30995393)
That seems pretty unlikely to me actually. As the models and training data get much bigger, I think the initial weights become less important (at least assuming your random weights have certain desirable statistical properties, which they do by construction usually).

There are no comments on this comment yet.