The quest to evolve neural networks through evolutionary algorithms

Re:The quest to evolve neural networks through evolutionary ... by halflings on Wednesday November 01, 2017 @11:22PM (#15607585)

Thanks! That link was really helpful. I guess some optimisation methods using a Hessian take those interaction effects into consideration, but they are too expensive and not used in practice.

> doesn't work for most optimisation problems - you get stuck in local optima.

This bit is not necessarily true. In practice, and given the high dimensionality, it is (apparently) quite hard to get stuck in local optima. [0]

[0] https://www.quora.com/How-come-neural-networks-dont-get-stuc...

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by randomsearch on Thursday November 02, 2017 @02:19PM (#15612856)

> This bit is not necessarily true. In practice, and given the high dimensionality, it is (apparently) quite hard to get stuck in local optima. [0]

In almost every optimisation problem I've encountered in practice, you do get stuck in local optima using simple techniques like SGD. Interaction effects are everywhere, and create local optima.

[ Parent | on HN ]