News for hackers. Stuff that matters.
Re:Machine learning spots natural selection at work in human... by paraschopra on Friday November 02, 2018 @05:37AM (#18361701)
I increasingly feel that deep learning needs to incorporate more ideas from evolution. Not just for parameter optimization but architecture discovery itself.

Imagine pitting neural networks in an adversarial environment (just like the real world). Under competitive pressure for limited food (computation resources), evolved neural network could start approaching optimal architectures that do the job but have no superfluous, pre-conceived notions of the (modeled) world. In fact, such evolved architectures could encode relevant notions of the world directly which we can learn about by reverse engineering evolved architecture.

This is closely related to the ideas from predictive processing which closely ties survival to prediction (as to predict your future states is to avoid getting dissipated). So I'm anticipating evolution or survival notions to come up in a big way in ML/Deep Learning in future.

Re:Machine learning spots natural selection at work in human... by rytill on Friday November 02, 2018 @10:35AM (#18363300)
This is currently very prolific and successful. You don’t have to imagine anything.

Latest popular paper on the specific application of evolution you’re talking about: https://arxiv.org/abs/1808.00193

Re:Machine learning spots natural selection at work in human... by DEADBEEFC0FFEE on Friday November 02, 2018 @06:48AM (#18362020)
Do you mean, like a GA of NNs, pretty sure that has been done many times.
Re:Machine learning spots natural selection at work in human... by Izkata on Friday November 02, 2018 @01:50PM (#18365117)
> Imagine pitting neural networks in an adversarial environment (just like the real world). Under competitive pressure for limited food (computation resources), evolved neural network could start approaching optimal architectures that do the job but have no superfluous, pre-conceived notions of the (modeled) world.

Sounds almost like these robots [0], which learned to lie to each other. They didn't use a neural net, though.

[0] http://discovermagazine.com/2008/jan/robots-evolve-and-learn...

Re:Machine learning spots natural selection at work in human... by ovi256 on Friday November 02, 2018 @06:21AM (#18361888)
>approaching optimal architectures that do the job but have no superfluous, pre-conceived notions of the (modeled) world

Usually, modelling efficiency is related to the amount of priors. The more (correct) priors, the faster a model learns, and from less data.

A prior-poor model just spends data learning those correct priors in the first place.

There are countless examples I could give to support intuition of this. There isn't, AFAIK, any theoretical work yet.

That the priors are correctly aligned with reality is vital. A model with fixed bad priors cannot unlearn them. If the priors are bad but not fixed, you're just spending time and data to correct those priors.

Re:Machine learning spots natural selection at work in human... by paraschopra on Friday November 02, 2018 @07:59AM (#18362353)
Yes, and evolution builds priors into organisms.