Re:Why AI lags behind the human brain in computational power
by TheDudeMan on Saturday November 06, 2021 @01:16PM
(#29131969)
Lost me at "Traditional neural nets have two input layers."
[ Parent
| on HN ]
Re:Why AI lags behind the human brain in computational power
by named-user on Saturday November 06, 2021 @02:12PM
(#29132436)
Yes, the input layer and .... The input layer?
Re:Why AI lags behind the human brain in computational power
by candiodari on Sunday November 07, 2021 @06:15PM
(#29144529)
There used to be a time, long ago, when neural networks had 2 layers: the input layer and the output layer, with full connections between the two. There's an old proof that, if you don't have nonlinearities there's an equivalent 2-layer net for any arbitrarily deep neural net.
So the real difference between modern "deep neural networks" and "neural networks" is not actually the depth, even though it is. It's just that neural networks, old definition, a 2 layer net can match any depth net, so you wouldn't use more. The real difference isn't the depth, but the nonlinearity (the tanh/sigmoid/relu operation).