For Google etc. it probably makes sense to pay top dollar for lots of ML PhDs (specialists).
A startup looking for opportunities engendered by ML is better off with a smart generalist. E.g. Dawson Whitfield of Logojoy was a designer. [1]
[0] https://twitter.com/patio11/status/936628610474983424?lang=e...
[1] https://www.indiehackers.com/podcast/038-dawson-whitfield-of...
Reading HN and following key people gives you the impression that this is an incredibly popular field - it is. In reality, though, it's a small field and lots of companies still don't use it, intend to use it, or have any idea how to use it. Alexnet was only 5 years ago. Faster-RCNN was only 2 years ago. Compared to, say, all programmers, statistically nobody is doing this.
Debugging a model is hard, but it's unlikely your efforts will be significantly more or less productive than a leading researcher aside from obvious things like noticing that the learning rate is too high or low. There's so much hand waving, even in top papers, that mostly we've tried stuff that looked like a good idea, and it turned out to be correct.
The thing is that a lot deep learning tasks don't require a fancy model. You can take VGG/Inception/ResNet and bodge them to fit a lot of real-world scenarios. Then the problems are mostly solved by standard machine learning intuition - making sure you have an appropriate train/test/validation set, a good loss function, etc.
Edit -
Here's an example: Halcon, a popular commercial image processing library has recently included a deep learning package due to repeated customer demand. It's not out yet, but it's most likely just going to be a wrapper for train/test using a basic image classification or object detection network. That's what the market wants. They don't care about GANs, they want something which will tell them if the image represents a valid product during QA 99% of the time (and often QA failures are pretty obvious - e.g. big crack in a bottle, or a deformed label).
You could build that kind of model with very little expertise and you'd beat most of the available image processing packages easily, with practically no work. The point is that an off-the-shelf model trained on company-specific data is likely to be state of the art on that problem (because who else has the dataset?) You don't care about 1000 classes in Imagenet, you want good product/bad product.
How rare is it to find a new problem domain these days? Most if not all problems tackled with deep learning have already been done in one form or the other. Deep learning is not yet much of an academic discipline, but accessible to anyone willing to put in the hours of study and practice.
Curious if anyone's done this: deep learning on DOM trees, to recognize portions of HTML pages. This has elements in common with both CNNs (where you want surrounding context - parents, ancestors, siblings, and descendants - to influence the recognition of a particular DOM element) and with RNNs (because elements may have an unbounded number of children, and potentially a recurring substructure), but I don't know offhand how you would combine them to solve the problem.
Anyone working on this? A quick literature search turned up nothing.
Seems like a really interesting idea to work on.
While I don't entirely agree with this assumption, if you are in business it's probably a "safer" assumption to make.
See how far they get. A talented guy can solve these within one year.
Many businesses have problems that an off the rack solution substantially solves it.
If AI is an integral part of my startup I would prefer to have someone with adequate experience in the founding team or very early hire.
90% of the work might not be super deep fundamental design decisions or debugging.
Yes, that guy can become a world class practitioner.
He was talked down to junior and paid as little as possible because that's what companies do.
"even for highly technical aspects like deep learning, fast.ai has shown that people with 1-year of coding experience can become world-class deep learning practitioners"
Yes, any Joe can train deep network with dozen lines of Keras. Sure, your startup can use off-the self models and tweak it a bit... That doesn't make you "world-class deep learning practitioner". If you are designing the network for new problem domain, there are thousands of decisions to make - everything from hyperparameters to network architecture to distributions in data. Making these decisions without having developed lots of intuition and good foundations is very hard. I often find usual developers without ML background and experience completely lost in these cases. Debugging a model that doesn't work is super hard. There are no IDEs, no breakpoints, no watches and in fact not even error messages. Its purely statistical debugging and probabilistic fixes.