Re:Differentiable Programming Mega-Proposal
by excessive on Saturday September 07, 2019 @06:29AM
(#20902940)
That's a fair point (and worth documenting as part of the library), but I'd be much more likely to implement pow(x, x) carefully than switch to a different language or compiler.
Re:Differentiable Programming Mega-Proposal
by joe_the_user on Saturday September 07, 2019 @03:38PM
(#20906003)
I appreciate the library approach and I basically agree with you. If anything, I wanted to point out that a normal general language is so general that adding differentiation into it would be highly costly - just this feature would require that every loop and every conditional return be watched (relative to pow(x,x), an even more challenging example is the Newton's method implementation of a continuous function using a loop).
An alternative would be creating a general purpose language just for this feature - an interesting though rather specialized project.
But this goes to question at hand, whether AD should be a library/DSL or whether it should be a primitive of a general purpose language. The thing is a general purpose language lets you present sorts of things as functions that can't be even dual numbers won't take the derivative of correctly - a dual scheme can't distinguish variable order loops from constant order loops.