News for hackers. Stuff that matters.
Re:Are deep neural networks dramatically overfitted? (2019) by samcodes on Monday April 05, 2021 @09:31AM (#26698679)
this is a good explanation of intrinsic dimension https://eng.uber.com/intrinsic-dimension/
Re:Are deep neural networks dramatically overfitted? (2019) by qPM9l3XJrF on Wednesday April 07, 2021 @12:11AM (#26720516)
Thanks. The implication seems to be that if you restrict the # of parameters so it's equal to intrinsic dimension, learning isn't possible, but it is possible with this random projections method. Wonder why. It seems like with both methods, the number of possibilities being explored is the same, but the higher-parameter model space is richer with solutions for some reason.