I think even if you avoid constructing features, you are basically doing a similar process where a single change in a hyper-parameter can have significant effects:
- internal structure of a model (what types of blocks are you using and how do you connect them, what are they capable of together, how do gradients propagate?)
- loss function (great results come only if you use a fitting loss function)
- category weights (i.e. improving under-represented classes)
- image/data augmentation (self-driving car won't work without significant augmentation at all)
- properly set-up optimizer
The good thing here is that you can automate optimization of these to a large extent if you have a cluster of machines and a way to orchestrate meta-optimization of slightly changed models. With feature engineering you just have to do all the work upfront, thinking what might be important, and often you just miss important parts of features :-(
curious, if there is any good quality open source project for this..
Further, I have repeatedly heard people who should know better, with very fancy advanced degrees, chant variants of "Deep Learning gets better with more data" and/or "Deep Learning makes feature engineering obsolete" as if they are trying to convince everyone around them as well as themselves that these two fallacious assumptions are the revealed truth handed down to mere mortals by the 4 horsemen of the field.
That said, if you put your ~10,000 hours into this, and keep up with the field, it's pretty impressive what high-dimensional classification and regression can do. Judea Pearl concurs: https://www.theatlantic.com/technology/archive/2018/05/machi...
My personal (and admittedly biased) belief is that if you combine DL with GOFAI and/or simulation, you can indeed work magic. AlphaZero is strong evidence of that, no? And the author of the article in this thread is apparently attempting to do the same sort of thing for self-driving cars. I wouldn't call this part of the field irrational exuberance, I'd call it amazing.