News for hackers. Stuff that matters.

A comment on An idea from physics helps AI see in higher dimensions. Parent comment

Re:An idea from physics helps AI see in higher dimensions by anonytrary on Friday January 10, 2020 @12:18AM (#22008506)
> One of the strangest things in AI, to me, is that you can average the weights of multiple different models to create a single model that’s better than any of the individuals.

I think this is one of the least strange things in AI. All you're doing is taking N overfitted models (unlikely to be overfit in the same way) and then asserting that the average of those predictions is probably not overfitted as much (regularization). Overfitting as a concept is not restricted to some number of dimensions.

[ Parent | on HN ]
Re:An idea from physics helps AI see in higher dimensions by MiroF on Friday January 10, 2020 @12:47AM (#22008625)
That would be averaging the predictions, not averaging the weights.

It doesn't really matter though, because the parent comment is nonsense and you can't just average the weights at the end and get a working neural net.

[ Parent | on HN ]
Re:An idea from physics helps AI see in higher dimensions by anonytrary on Friday January 10, 2020 @12:54AM (#22008656)
I just assumed he meant averaging the predictions but said it wrong.
[ Parent | on HN ]