News for hackers. Stuff that matters.

A comment on Matrix Calculus for Deep Learning. Parent comment

Re:Matrix Calculus for Deep Learning by SpaceManNabs on Friday November 29, 2019 @02:42AM (#21662795)
Not necessarily true. The polynomial time for the escape of saddle points by using stochastic noise is pretty practical.
[ Parent | on HN ]
Re:Matrix Calculus for Deep Learning by p1esk on Friday November 29, 2019 @08:37AM (#21664261)
How in the world is this "pretty practical"? It would be if people used this theorem to come up with an idea of SGD, but that's not what happened. SGD appeared as a way to overcome the practical constraint of computing full GD. Not to mention that "polynomial time" is meaningless to any practitioner.
[ Parent | on HN ]
Re:Matrix Calculus for Deep Learning by SpaceManNabs on Wednesday December 04, 2019 @04:59PM (#21707652)
it did come with a new online update for orthogonal tensor decomposition using higher order moments and with comments on NP-hardness for 4th order and higher.

In addition, it came with tricks with how much noise to inject in certain situations. "How much noise do you need is enough to escape?" which is pretty practical

[ Parent | on HN ]
Re:Matrix Calculus for Deep Learning by nightcracker on Friday November 29, 2019 @03:20AM (#21662926)
Sorry, my statement was too strong. I edited it to "usually".
[ Parent | on HN ]