We are inventing new "real world" benchmarks to try and counteract this (MS COCO, dialog datasets, the big flickr datasets, translation generally), but many approaches from the 90s were clearly right mathematically (as Jurgen says) and just needed more data fuel. So it is obvious to go back and find interesting ideas that didn't get their due as long as proper attribution is given. Profs also have their pet projects that didn't quite pan out, and often want to breathe new life into a cool idea.
These things are only easy to half-assed cobble together in hindsight AND/OR if you have expertise - having the knowledge and know how to input conditional information, interpreting deep networks as modeling joint probability distributions, etc. is just as much algorithmic design as any other task in graphical modeling, statistics etc. Slapping a big deep convnet (or feedforward net) on new datasets IS easy, and usually not interesting scientifically, but also doesn't get published and is reserved for the blogosphere or bad ArXiV papers.
Incremental progress is 0.5% performance gains in major benchmarks like ImageNet etc. - company PR (and university PR as well) will crow about this but no one in academia really cares unless it is accompanied by interesting scientific ideas or fundamental questions being answered.
What I'm curious about is how such hype cycles affect our understanding of whether this shit actually works.
As far as I can tell, the "great advance" attributed to deep learning is doing benchmarks better than other methods. Which seems to have the result of doing the same old machine learning tasks more accurately than previously.
The main thing we hear about is more "edgy" stuff like learning video game by oneself, describing pictures with phrases or giving answers to "philosophy". But since these applications are easy to cobble together in a half-assed fashion, is there really more here than incremental progress?
I wouldn't know one way or the other, so I'd like to find out.