The quest to evolve neural networks through evolutionary algorithms

Re:The quest to evolve neural networks through evolutionary ... by maaaats on Monday October 30, 2017 @09:41AM (#15584905)

I think the multiobjective-part is very important. In other algorithms, you often have to specify the solution-space before hand. For instance, when optimizing between lightweight and strength, one would have to beforehand say how to weight those two properties. (f = 100s - 5w for instance). This throws away a whole dimension in your search space. For EAs, you can let it roam free, and select the bests tradeoffs from the pareto-set after the algorithm is done.

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by sevensor on Monday October 30, 2017 @09:53AM (#15584972)

Yes, exactly! And it gets even better. With a good MOEA, you can keep track of your evaluations, do a short run, decide your objectives don't really express your understanding of the problem, and seed your old evaluations into a run with new objectives. This means you can change objectives without repeating computationally expensive model runs.

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by zardo on Monday October 30, 2017 @03:16PM (#15588008)

The IMO, most interesting work in EA is in 'search space illumination', MAPElites, and it's variations are a great example.

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by sevensor on Monday October 30, 2017 @09:36PM (#15590529)

Thanks for pointing this out. I had a look at http://www.evolvingai.org/files/1504.04909v1.pdf. It's an interesting read, although I'm not sold on the value of feature space exploration. I think it muddies the waters around the problem of balancing optimization with diversity, by lumping decisions and constraints together. What's to stop it from over-sampling regions of the decision space that have a steep gradient in the other features?

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by zardo on Tuesday October 31, 2017 @01:29PM (#15594923)

What's to stop the discovered solutions from being similar? Nothing but the dimensions of interest. If the best solutions with respect to features a, b, and (a&b) are all the same, then it does not seek different solutions.

This may indicate you chose reduntant features though.

Here's an interesting application that provides some motivation. https://arxiv.org/abs/1407.3501

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by sevensor on Tuesday October 31, 2017 @05:02PM (#15596543)

Regarding the similarity of solutions:

If, in a particular region, your model outputs are particularly sensitive to the inputs, that region can be overrepresented in a feature map. What I mean by this is that we cannot effectively choose between the solutions in this region -- the region is small compared to our ability to control inputs. Overrepresenting such a region in a population-based search starves the rest of the problem space of the algorithm's attention.

Thanks for the link, by the way -- this is a particularly interesting application.

[ Parent | on HN ]