News for hackers. Stuff that matters.
Re:Why neural networks struggle with the Game of Life (2020) by jxy on Friday May 17, 2024 @02:05PM (#40392604)
> there are fewer local minima in a higher dimension

Is it actually proven, or another hypothesis? What is the reason behind this?

Re:Why neural networks struggle with the Game of Life (2020) by jcrites on Friday May 17, 2024 @02:40PM (#40392969)
Just reasoning about this from first principles, but intuitively, the more dimensions you have, the more likely that you are to find a gradient in some dimension. In an N-dimensional space, a local minimum needs to be a minimum in all N dimensions, right? Otherwise the algorithm will keep exploring down the gradient. (Not an expert on this stuff.) The more dimensions there are, the more likely it seems to be that a gradient exists down to some greater minimum from any given point.