Re:Do Machine Learning Models Memorize or Generalize?
by timy2shoes on Thursday August 10, 2023 @05:34PM
(#37082094)
It's interesting that the researchers chose example problems where the minimum norm solution is the best at generalization. What if that's not the case?
[ Parent
| on HN ]
Re:Do Machine Learning Models Memorize or Generalize?
by mashygpig on Thursday August 10, 2023 @07:20PM
(#37082999)
Yea, this is what’s really going on here and feels like it’s been shrouded in language to make it seem more grandiose. That being said, I would believe generalization to occur from minimum norm solutions in some sense, but whether that corresponds to minimum norm weights or not is a different question, and one you probably won’t know a priori (not to mention even knowing which norm to choose).