Re:Ask HN: Is deep learning obsession in college ill founded?
by deuslovult on Monday May 11, 2020 @02:03PM
(#23145062)
Logistic regression is exactly a NN with no hidden layers and a sigmoid activation function. A feedforward NN with additional layers is strictly more expressive than logistic regression.
[ Parent
| on HN ]
Re:Ask HN: Is deep learning obsession in college ill founded?
by mattkrause on Monday May 11, 2020 @02:53PM
(#23145615)
Yes! The million dollar question is how much of that expressivity is actually required.
In many papers, the "baseline" logistic regression model is very stripped down: y~logit(.) but the neural network has had its expressiveness optimized in various ways. People aren't comparing against a 3 layer feedfoward network; there's augmentation and pre-training, architecture search and special learning schemes.
My claim is that if you want to claim that a problem needs the expressivity that (only) a neural network provides, you ought to be devoting a great deal of effort to the logistic regression model too. Make it a steelman, rather than a strawman, if you will.