News for hackers. Stuff that matters.

A comment on Computing Higher Order Derivatives of Matrix and Tensor Expressions [pdf]. Parent comment

Re:Computing Higher Order Derivatives of Matrix and Tensor E... by jkam on Friday November 16, 2018 @03:04PM (#18471554)
What kind of work are you referring to when you say higher-order SGD may _now_ be feasible for deep learning? I only find results that try to approximate second order information.
[ Parent | on HN ]
Re:Computing Higher Order Derivatives of Matrix and Tensor E... by bmc7505 on Saturday November 24, 2018 @10:44PM (#18525359)
Not sure what you mean. The paper above claims 1000x speedups for computing second-order derivatives. Have not tested their claims, but was speculating that such an improvement, if true, would make computing hessians for small networks fesiable. This is what I am referring to.
[ Parent | on HN ]