News for hackers. Stuff that matters.
Re:There Will Be a Scientific Theory of Deep Learning by charcircuit on Saturday April 25, 2026 @03:23AM (#47899444)
>you'd need every single one of them, millions up millions of them, to be all zero

If they were all correlated with each other that does not seem far fetched.

Re:There Will Be a Scientific Theory of Deep Learning by imtringued on Saturday April 25, 2026 @05:29AM (#47900028)
Ok but it's already known that you shouldn't initialize your network parameters to a single constant and instead initialize the parameters with random numbers.
Re:There Will Be a Scientific Theory of Deep Learning by charcircuit on Saturday April 25, 2026 @06:35AM (#47900342)
The model can converge towards such a state even if randomly initialized.
Re:There Will Be a Scientific Theory of Deep Learning by hodgehog11 on Saturday April 25, 2026 @08:16AM (#47900830)
Both you and the comment above are correct; initializing with iid elements ensures that correlations are not disastrous for training, but strong correlations are baked into the weights during training, so pretty much anything could potentially happen.