Re:Gödel Machine
by jimfleming on Thursday March 28, 2019 @05:24PM
(#19516786)
Hyperparameter optimization (including architectures) is not really meta-learning. Meta-learning, also known as "learning to learn", is more like MAML[0], RL2[1], L2RL[2], etc.
0. https://arxiv.org/abs/1703.03400
Meta-learning, where you also learn hyper-parameters (including potentially the architecture) are closer to that, see for example:
- Google Vizier: https://ai.google/research/pubs/pub46180
- Microsoft neural architecture search: https://github.com/Microsoft/nni
But this is still far from a true Godel machine.