> learn gradient descent by gradient descent.
As far as 'learning gradient descent by gradient descent' goes, I think it's still an open question if a tiny RNN actually can improve meaningfully over ADAM etc :) I don't recall the paper showing any wallclock times, which suggested to me that the RNN was way slower even if it trained faster. The individual parameter adjustments are so low in the hierarchy of actions that the value may be minimal compared to higher up like architecture design.
Re:Tool AIs want to be agent AIs
by AlexCoventry on Thursday December 22, 2016 @06:40AM
(#13236601)
What paper are you referencing?
I assume this https://arxiv.org/abs/1606.04474 (since the title is basically the quoted text)
I especially liked this passage. It connects many ideas onto one theme:
> CNNs with adaptive computations will be computationally faster for a given accuracy rate than fixed-iteration CNNs, CNNs with attention classify better than CNNs without attention, CNNs with focus over their entire dataset will learn better than CNNs which only get fed random images, CNNs which can ask for specific kinds of images do better than those querying their dataset, CNNs which can trawl through Google Images and locate the most informative one will do better still, CNNs which access rewards from their user about whether the result was useful will deliver more relevant results, CNNs whose hyperparameters are automatically optimized by an RL algorithm will perform better than CNNs with handwritten hyperparameters, CNNs whose architecture as well as standard hyperparameters are designed by RL agents will perform better than handwritten CNNs… and so on. (It’s actions all the way down.)
This is the general trend. Going meta, through RL. Optimize the optimizer, learn gradient descent by gradient descent.