Re:The limitations of gradient descent as a principle of bra...
by simonster on Sunday June 10, 2018 @02:08PM
(#17279345)
tl;dr ordinary gradient descent is sensitive to the parametrization of the problem but natural gradient is not. This is an important fact (and one that is fairly well-known within the ML community), but it is not totally clear to me why it should be particularly relevant to neuroscience.
[ Parent
| on HN ]
Re:The limitations of gradient descent as a principle of bra...
by bjourne on Sunday June 10, 2018 @08:34PM
(#17280924)
Isn't it only the trajectory that is sensitive to the parametrization? Afaik, the location of the minimas do not depend on it.