The quest to evolve neural networks through evolutionary algorithms

Re:The quest to evolve neural networks through evolutionary ... by sevensor on Monday October 30, 2017 @09:34AM (#15584858)

One thing I don't see mentioned here is flexibility. You can plug any ugly old thing into an evolutionary algorithm and let it run. Give me a working computational model at breakfast time and I'll have runs going by lunch. Cranky ancient FORTRAN that requires you to write a new input file for every evaluation? No problem. You have to compile the inputs into the model to make it run? Fine. Badly scaled inputs or outputs? EA doesn't care. More than one objective? Great! Population-based search is a natural fit for multiple objective optimization.

As long as you're clear on what the decisions and objectives are, and you're able to run the model yourself, layering evolutionary optimization on top is easy.

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by maaaats on Monday October 30, 2017 @09:41AM (#15584905)

I think the multiobjective-part is very important. In other algorithms, you often have to specify the solution-space before hand. For instance, when optimizing between lightweight and strength, one would have to beforehand say how to weight those two properties. (f = 100s - 5w for instance). This throws away a whole dimension in your search space. For EAs, you can let it roam free, and select the bests tradeoffs from the pareto-set after the algorithm is done.

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by sevensor on Monday October 30, 2017 @09:53AM (#15584972)

Yes, exactly! And it gets even better. With a good MOEA, you can keep track of your evaluations, do a short run, decide your objectives don't really express your understanding of the problem, and seed your old evaluations into a run with new objectives. This means you can change objectives without repeating computationally expensive model runs.

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by zardo on Monday October 30, 2017 @03:16PM (#15588008)

The IMO, most interesting work in EA is in 'search space illumination', MAPElites, and it's variations are a great example.

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by sevensor on Monday October 30, 2017 @09:36PM (#15590529)

Thanks for pointing this out. I had a look at http://www.evolvingai.org/files/1504.04909v1.pdf. It's an interesting read, although I'm not sold on the value of feature space exploration. I think it muddies the waters around the problem of balancing optimization with diversity, by lumping decisions and constraints together. What's to stop it from over-sampling regions of the decision space that have a steep gradient in the other features?

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by zardo on Tuesday October 31, 2017 @01:29PM (#15594923)

What's to stop the discovered solutions from being similar? Nothing but the dimensions of interest. If the best solutions with respect to features a, b, and (a&b) are all the same, then it does not seek different solutions.

This may indicate you chose reduntant features though.

Here's an interesting application that provides some motivation. https://arxiv.org/abs/1407.3501

[ Parent | on HN ]

Re:The quest to evolve neural networks through evolutionary ... by sevensor on Tuesday October 31, 2017 @05:02PM (#15596543)

Regarding the similarity of solutions:

If, in a particular region, your model outputs are particularly sensitive to the inputs, that region can be overrepresented in a feature map. What I mean by this is that we cannot effectively choose between the solutions in this region -- the region is small compared to our ability to control inputs. Overrepresenting such a region in a population-based search starves the rest of the problem space of the algorithm's attention.

Thanks for the link, by the way -- this is a particularly interesting application.

[ Parent | on HN ]