Re:Exploring Weight Agnostic Neural Networks
by kolar on Friday August 30, 2019 @05:28AM
(#20837036)
Genetic Programming tries to do almost the same: ie. generate random programs/expression trees etc and then select the best one. NN gives you two adventages over that: you can optimize weights and do it at scale with current hardware - without those two I can't see the difference.
The work here is in a similar vein as the Lottery Ticket Hypothesis ( https://arxiv.org/pdf/1803.03635.pdf ), which found that deep nets (for vision) contain discriminative sub-networks at initialization time (due to random initialization), before training ever starts.
While the authors of this work on architecture search say they hope to inspire the discovery of new architectures, a more immediately striking result of their work is that they get functioning systems from doing something “stupid” (i.e. not optimizing weights).