Re:DeepMind’s New Language Model, Chinchilla
by eigenvalue on Monday April 11, 2022 @05:33PM
(#30995393)
That seems pretty unlikely to me actually. As the models and training data get much bigger, I think the initial weights become less important (at least assuming your random weights have certain desirable statistical properties, which they do by construction usually).
[ Parent
| on HN ]
There are no comments on this comment yet.