News for hackers. Stuff that matters.
Re:There's No Fire Alarm for Artificial General Intelligence by Eliezer on Sunday October 15, 2017 @07:26AM (#15476704)
Now that we know how to initialize the weights so as to have the layer activations be something like sane, yes, we can use sigmoid/tanh. If you don't know modern clever ways of initializing weights then multi-layered sigmoid/tanh causes your activations and gradients to die out fast in deep networks, and ReLU is a godsend.

There are no comments on this comment yet.