But seriously: I think you're overstating the case. I've enjoyed watching the fast.ai videos, and I think deep learning is a clear choice for some areas right now. Many others, no.
If you're working in a data-limited domain, in particular (i.e. most of them) there are probably better first choices. I know plenty of folks who have used deep learning to achieve no better than comparable results to the simpler methods they were using before.
But yes, the initial engineering costs of a deep learning system have come down a lot further than I thought before I watched your videos.
Most of the hate for deep learning on HN is usually against using it in titles as clickbait. (i.e. just simply taking a pretrained/predefined model and changing the source dataset without any optimization)
The in-depth explanations of mechanics discussed in deep learning as noted in fast.ai courses and this submission do justify the use of the deep learning moniker, though.
Deep learning is great if you're dealing with a hugely dimensional problem, and you have the data to train the model. If one of those things is not true (and usually, one of those things is not true), you're better off starting simple.
So even if I didn't have the data to train the model I had enough data to bootstrap the process and sometimes that's all you need.
It bothers me how often otherwise rational individuals continue to be susceptible to survivorship bias.
Of course you'd only hear about the successes, the failures are either embarrassed or know better than to tell you their story.
The people who tried it for several months, saw accuracy cap out at something like 60-70% for processes that need something like >90% confidence to justify expenses, and which proceed to get ignominiously shifted into a different team (or fired) for building what the higher-ups view as a massive waste of money at your salary and the other engineers see as a data-guzzling black box. No, these guys aren't likely to tell their story. Or, at least THIS story.
The story you instead get from these guys is how
>After having had some fun times doing deep learning and data analysis, I'm excited get into the big new field of $THING_THAT_WAS_HIRING
and all the other dross that modern vocal programmers use to mask any possible scent of failure and actual on-the-job difficulty experienced.
What I always try to do is to get a feel for the problem space trying different methods with as little investment as possible. That way you will - once you decide to go full power after a certain solution method - at least have a feeling that you are on the right path.
Dogmatically trying to shoehorn every problem into the toolset that you know how to use is a way to stay reasonably productive but it rarely leads to optimal outcomes, sometimes you simply have to learn how to use a new tool in order to get to the maximum.
I'm the last person to jump on new bandwagons, still have a dumb phone, don't use facebook and still run my own mailserver. Even so, when a tool has a significant and most importantly measurable advantage compared to the tools I'm already familiar with I'll adapt.
That's fine, but the thing is they once again plan to apply it on their badly maintained data warehouses (now often somewhere in their dusty hdfs stack) to build traditional predictive models. Customer churn, next best offer, propensity modeling, that sort of thing.
I try to keep telling them that with structured data sets, and simple classification problems, a random forest or God forbid, a logistic regression model, would do just as well given that you spend some solid time in feature cleaning and preprocessing. Am I wrong in this setting? Perhaps I should also jump on the deep learning bandwagon and start selling 10 layer binary Classification networks :).
Not disagreeing with you, but I do fear that many traditional industries will be jumping on board pretty soon without any of the actual use cases to warrant deep learning.
The truth is that an imagenet pre trained network will often work astonishingly well. The only other mark against deep learning is computational complexity, which I expect to go away soon (yours truly ;) )
Could you elaborate more on what has changed recently in the theoretical grounding of deep learning?
We've seen, in the last year or two, interesting results in nearly every area of theoretical research of deep learning, including generalization, optimization, generative modelling, bayesian models, and network architecture.
It is possible that at some point to get to the last .1% or so I may have to embed features that were engineered but right now there is no indication that that will be the case.
The key insight here, to me, is that deep learning saves a lot of time, as well as being more accurate. I hear very frequently people say "I'll just start with something simple - I'm not sure I even need deep learning"... then months later I see that they've built a complex and fragile feature engineering system and are having to maintain thousands of lines of code.
Every time I've heard from someone who has switched from a manual feature engineering approach to deep learning I've heard the same results as Jacques found in his lego sorter: dramatic improvements in accuracy, generally within a few days of work (sometimes even a few hours of work), with far less code to write and maintain. (This is in a fairly biased sample, since I've spent a lot of time with people in medical imaging over the past few years - but I've seen this in time series analysis, NLP, and other areas too.)
I know it's been trendy to hate on deep learning over the last year or so on HN, and I understand the reaction - we all react negatively to heavily hyped tech. And there's been a strong reaction from those who are heavily invested in SVMs/kernel methods, bayesian methods, etc to claim that deep learning isn't theoretically well grounded (which is not really that true any more, but is also beside the point for those that just want to get the best results for their project.)
I'd urge people that haven't really tried to built something with deep learning to have a go, and get your own experience before you come to conclusions.