Differentiable Programming Mega-Proposal

Re:Differentiable Programming Mega-Proposal by seanmcdirmid on Thursday September 05, 2019 @10:22PM (#20892497)

There is a derivative, but since the function is non continuous, the derivative will likewise be messed up (but just around x=59). You really don’t want to be climbing a gradient around non continuous functions!

[ Parent | on HN ]

Re:Differentiable Programming Mega-Proposal by joe_the_user on Friday September 06, 2019 @02:26AM (#20893528)

The function has a derivative by some notions of derivative.

But the function's derivative can't be derived by an application of the chain rule and the know derivatives of primitive functions, which is what Algorithmic/automatic differentiation ultimately does (though it does this at run time, not compile time, since ordinary, symbolic differentiation explodes in memory for a complicated functions).

Also:

The continuous function

    int f(int x) {
        if(x > 2) 
            return x + x;
        else 
            return x*2;
    }

Is not automatically or symbolically differentiable when represented that way.

[ Parent | on HN ]

Re:Differentiable Programming Mega-Proposal by Certhas on Friday September 06, 2019 @08:38AM (#20895066)

I believe a source to source differentiator could deal with all these (where well defined of course), e.g.:

https://github.com/FluxML/Zygote.jl

    julia> fs = Dict("sin" => sin, "cos" => cos, "tan" => tan);
    
    julia> gradient(x -> fs[readline()](x), 1)
    sin
    0.5403023058681398

[ Parent | on HN ]