Re:From automatic differentiation to message passing [video]
by proofofconcept on Thursday August 15, 2019 @01:43PM
(#20707612)
>there is a follow up paper to the DL via Hessian-free optimization paper by James Martens that develops a variant of AD which calculates a special curvature quantity which is useful for efficient second order optimization
Deep Learning via Hessian-free Optimization: http://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pd...
Optimizing Neural Networks with Kronecker-factored Approximate Curvature: http://arxiv.org/abs/1503.05671
James Martens' list of publications with links to sample code for the above two papers, slides/condensed conference versions, etc: http://www.cs.toronto.edu/~jmartens/research.html
Pretty neat stuff