Notes on Weight Initialization for Deep Neural Networks

Re:Notes on Weight Initialization for Deep Neural Networks by axiom92 on Tuesday May 21, 2019 @02:01PM (#19972840)

> They are large, but far from overflowing.

Sure, but isn't large relative? Sure you can make them overflow in octave as well, given enough layers. Which brings us to next point :-)

> And this corresponds to a network of deep 100, which is not a realistic scenario.

Actually deep 100 is not unrealistic at all these days! https://arxiv.org/abs/1611.09326

[ Parent | on HN ]

Re:Notes on Weight Initialization for Deep Neural Networks by L2R on Wednesday May 22, 2019 @01:19PM (#19983271)

There are approaches to ensure parameters remain stable despite the depth (selu, for example).

[ Parent | on HN ]