Re:There Will Be a Scientific Theory of Deep Learning
by charcircuit on Saturday April 25, 2026 @06:35AM
(#47900342)
The model can converge towards such a state even if randomly initialized.
[ Parent
| on HN ]
Re:There Will Be a Scientific Theory of Deep Learning
by hodgehog11 on Saturday April 25, 2026 @08:16AM
(#47900830)
Both you and the comment above are correct; initializing with iid elements ensures that correlations are not disastrous for training, but strong correlations are baked into the weights during training, so pretty much anything could potentially happen.