Re:What's wrong with deep learning? (2015) [pdf]
by aab0 on Wednesday July 13, 2016 @10:34PM
(#12091321)
A lot of 2015-2016 work can be seen as addressing the latter two. For memory, all the work on neural programming, soft and hard attention, content-addressable memory and 'memory networks'. For unsupervised learning, adversarial networks have spawned a whole bunch of papers with OpenAI's latest batch being quite exciting.
For each problem, he explores some salient ideas or ways to address the issue.
TLDR:
* Theory: We don't always have good explanations for why it works.
* Reasoning: Stick a CRF on top of a Deep Net
* Memory: We need a "hippocampus". Memory networks, neural embeddings.
* Unsupervised Learning: How do we speed up inference in a generative model? Sparse autoencoders, sparse models...
For those who could use an overview of neural nets and how some of them work, this may be useful: http://deeplearning4j.org/neuralnet-overview.html