News for hackers. Stuff that matters.
Re:There Will Be a Scientific Theory of Deep Learning by geokon on Saturday April 25, 2026 @04:15AM (#47899667)
normally more parameters leads to overfitting (like fitting a polynomial to points), but neural nets are for some reason not as susceptible to that and can scale well with more parameters.

Thats been my understanding of the crux of mystery.

Would love to be corrected by someone more knowledgable though

Re:There Will Be a Scientific Theory of Deep Learning by hodgehog11 on Saturday April 25, 2026 @04:31AM (#47899736)
This absolutely was the crux of the (first) mystery, and I would argue that "deep learning theory" really only took off once it recognized this. There are other mysteries too, like the feasibility of transfer learning, neural scaling laws, and now more recently, in-context learning.