Re:Why neural networks struggle with the Game of Life (2020)
by WithinReason on Friday May 17, 2024 @10:04AM
(#40390040)
I think you're right, but the issue might be local minima which a better optimiser wouldn't help with much. A reason a larger network might work better is that there are fewer local minima in a higher dimension too.
[ Parent
| on HN ]
Re:Why neural networks struggle with the Game of Life (2020)
by jxy on Friday May 17, 2024 @02:05PM
(#40392604)
> there are fewer local minima in a higher dimension
Is it actually proven, or another hypothesis? What is the reason behind this?
Re:Why neural networks struggle with the Game of Life (2020)
by jcrites on Friday May 17, 2024 @02:40PM
(#40392969)
Just reasoning about this from first principles, but intuitively, the more dimensions you have, the more likely that you are to find a gradient in some dimension. In an N-dimensional space, a local minimum needs to be a minimum in all N dimensions, right? Otherwise the algorithm will keep exploring down the gradient. (Not an expert on this stuff.) The more dimensions there are, the more likely it seems to be that a gradient exists down to some greater minimum from any given point.