News for hackers. Stuff that matters.
Re:Self-Normalizing Neural Networks by unixpickle on Tuesday June 13, 2017 @04:41PM (#14548442)
I'm not sure I see why tanh couldn't be used to the same effect. If you use 1.6*tanh(x) as your activation function, it pushes small variances higher and high variances lower and gets you to a variance of ~1 after many layers. Obviously not as rigorous, just an observation.

There are no comments on this comment yet.