Deep learning is already a Scruffy fad. It basically says, "Hey, let's use a really huge hypothesis space of circuits that often includes a heavy prior towards convolutions." Gradient descent is a Neat principle, but the whole point of things like improved training methods, new objective functions, and convolutions was to deal with the exploding-gradient problem.
Deep learning didn't come up with its own Neat principle, it invented Scruffy methods to apply a Neat principle to a really fucking huge hypothesis space, so long as you've got a pretty big dataset.
I suspect the reason why deep learning has done so poorly in my domain is that the underlying data is a result of things that are very poorly abstracted as a "function". We have lots of discrete events, stateful buffering, hard non-linearities, discontinuities, numerical bounds, etc. It's more like learning business rules and physical process design than learning a mathematical function. This is part of the reason I don't see deep learning being a holistic solution for self driving cars...once you get past sensory perception and simple 2d path planning, driving is more of a rule based process than anything.
That being said, ML tends to be a pretty niche technique for us anyway. If a process and its components are well known and understood, we tend towards solutions that come from Operations Research over Machine Learning. It is only when things are poorly understood that we use ML (example: predicting product demand fluctuations based on media coverage or predicting truck arrival times given severe weather patterns and traffic backups). PGMs do really well here, but are far more difficult to understand, formulate, and train...for most tasks Random Forests are almost always Good Enough(TM).
Understandable models with clear intervention points are what most businesses seem to need once you get to digging around in their operations, customer and sales data.
Even taking data size out of the picture, functionally it is not there yet for most tasks. Maybe it will be in the future, but the big problem with it is that with n neurons, you have n^n possible topologies, and finding the right neural topology is a major optimization problem that we're only barely learning basic human heuristics for.
I'm willing to bet the deep learning thing is just one more Neat fad that will eventually cause disillusionment at its lack of results, reverting us back to the Scruffy view that intelligence is far too complex to be described holistically by small sets of simple algorithms. The great thing about the Scruffy philosophy is that it isn't derogatory...deep learning will always have a place as a tool in its tool set. It merely doesn't hold unreasonable expectations.