News for hackers. Stuff that matters.
Re:Differentiable Programming Manifesto by kragen on Wednesday November 13, 2019 @03:49PM (#21529379)
Well, successive-approximation mathematical optimization algorithms are sort of inherently anytime, in a way: you can stop after any integer number of iterations and use the approximate solution. Some of them fail to be anytime in practice, like SOR (“successive over-relaxation”) where you usually quit after like six iterations but each iteration takes a long time. And autodiff makes it a lot easier to apply mathematical optimization, in particular variants of gradient descent, one popular use of which is training deep ANNs.

Is that what you meant, or are you talking about something else?

Re:Differentiable Programming Manifesto by sitkack on Wednesday November 13, 2019 @04:58PM (#21530069)
This is it.

I think AD can be used to train a DNN, the DNN in turn can be used to power an anytime algorithm such that you can have systems that can not only respond on different time scales, they are also robust in the face of sensor noise, adversarial sensors or broken actuators.