Genetic algorithm: Shift parameters in a batch of random ways, evaluate using fitness function, keep the best.
There is nothing more fundamentally 'creative' about one than the other; these are both just attacks on the problem "here is this function, find parameter settings that get higher scores in it".
As you say, GAs evolve parameters to the fitness function. But GP does, in fact, evolve programs.
They are very distinct schools of work that evolved independently. Some other major schools are Learning Classifier Systems, Evolution Strategies and Evolutionary Programming.
The high-level name these days is Evolutionary Computation. In turn typically lumped under Nature-Inspired Computing, or sometimes Metaheuristics.
The entire article repeatedly relates everything in the context of what DL accomplishes and only introduces EC as an better optimized version.
Which doesn't lead one to think they are significantly different approaches but variations or tweaked versions of the same general thing...
This is just wrong.
But that's just my guess. And even so, I am not terribly convinced that would lead to astonishingly new activation functions that are better than the known ones.
We've had "learning-to-learn" algorithms for a few years now. Including LSTMs that can be learn the gradients for other LSTMs, and deep-RL algorithms that can optimize neural networks.
Its hard to say its inventing new things... there is still a clear goal - a loss - and we are optimizing it; poorly in the case of genetic algorithms.
I would say genetic methods are simply poorly described reinforcement learning problems, which means that there is a 1-to-1 mapping between Deep Learning and Genetic Algorithms.
PS. Making a distinction between "Genetic Algorithms" and "Genetic Programming" is like calling Deep Learning "Differential Programming" -- changing the name of a thing does not change the thing itself.
Also note that in the 90s things produced by genetic programming went patented, because they were novel algorithms.
An episode in this case is an iterative call to the Deep RL system until it outputs <STOP> at which step you give it a reward (negative number of collisions say), and before that you can send the actions to your stack machine.
Once you're satisfied with the final number, just concatenate all the produced actions to get your final program.
I just don't see a fundamental difference, especially once you start doing things like Asynchronous Actor-Critic et al. And then you can start doing Monte Carlo Tree Search on top of that since you have a simulator available...
http://www.genetic-programming.org/combined.php
Here's a fun article from Popular Science on Koza and his "invention machine" that readers might enjoy:
https://www.popsci.com/scitech/article/2006-04/john-koza-has...
The difference is that for a GA, that can be any function, whereas for NNs it must be decomposable into a sum of partial objectives, and differentiable. Moreover, the result of training an NN is a function mapping a vector to a vector, whereas the result of training a GA could be an object drawn from any search space.
Most DL work parameter tunes a structure that is hacked by human guesswork.
The two approaches have strengths and weaknesses, and in no way does this make EC in general superior to DL, but perhaps different.
What utter nonsense. Genetic Algorithms do exactly the same thing that Deep Learning methods do: optimize a function for a particular criterion. Genetic Algorithms are useful when taking gradients is not viable, as with RL methods - and RL methods can use Deep Learning! Seriously misleading.
Also, this: "Remarkably, although several human-designed LSTM variations have been proposed, they have not improved performance much—LSTM structure was essentially unchanged for 25 years. Our neuroevolution experiments showed that it can, as a matter of fact, be improved significantly by adding more complexity, i.e. memory cells and more nonlinear, parallel pathways." - this is not at all a new idea? How can you even pretend what you're doing is novel.