News for hackers. Stuff that matters.

A comment on A week-long programming retreat. Parent comment

Re:A week-long programming retreat by chillee on Monday March 05, 2018 @12:58PM (#16522903)
The things that have most improved training neural networks since you left were: 1. Smart (Xavier/He) initialization 2. ReLU activations 3. Batch normalization 4. Residual connections

The GPUs and dataset size were definitely very important though.

[ Parent | on HN ]
Re:A week-long programming retreat by The_suffocated on Monday March 05, 2018 @01:48PM (#16523286)
Thanks for the info. I may want to keep up with the recent advances some day.
[ Parent | on HN ]