News for hackers. Stuff that matters.

A comment on Differentiation for Hackers.

Re:Differentiation for Hackers by Bostonian on Saturday June 22, 2019 @08:41AM (#20249306)
What do automatic differentiation tools do for functions with integer arguments?
[ Parent | on HN ]
Re:Differentiation for Hackers by osipov on Saturday June 22, 2019 @09:39AM (#20249506)
Great question. Both TensorFlow and PyTorch pretty much do nothing https://github.com/tensorflow/tensorflow/issues/20524
[ Parent | on HN ]
Re:Differentiation for Hackers by mcabbott on Saturday June 22, 2019 @10:04AM (#20249604)
That seems strange, if I'm reading it correctly.

In the author's packages, integers which label dimensions etc. have gradient `nothing`, but arrays which happen to contain integers do not signal anything:

    julia> Zygote.gradient((x,d)->abs(sum(sin,fft(x,d))), [2 2; 0 0], 2)
    (Complex{Float64}[-0.346356+0.0im 1.65364+0.0im; -2.0+0.0im 0.0+0.0im], nothing)
[ Parent | on HN ]
Re:Differentiation for Hackers by bmc7505 on Saturday June 22, 2019 @10:51AM (#20249892)
It is possible to do backprop with integers: https://arxiv.org/abs/1802.04680
[ Parent | on HN ]