Re:Automatic Differentiation in Machine Learning: A Survey [...
by sgillen on Monday November 19, 2018 @09:26PM
(#18491422)
Can you elaborate? What makes you think that nilpotence is under appreciated?
[ Parent
| on HN ]
Re:Automatic Differentiation in Machine Learning: A Survey [...
by adamnemecek on Monday November 19, 2018 @09:43PM
(#18491478)
Well in this case, dual numbers have this nilpotent constant epsilon that’s analogous to i such that epsilon ^ 2 = 0. It’s the basis of automatic differentiation. I don’t fully understand it yet, but it seems that nilpotence is useful for smoothness and differentiability.
Re:Automatic Differentiation in Machine Learning: A Survey [...
by gugagore on Monday November 19, 2018 @09:54PM
(#18491510)
I think you are thinking about this the wrong way. nilpotence doesn't relate to smoothness (of what) and differentiability (of what). Dual numbers happen to, the way I see it, model forward-mode differentiation. There are a couple of ways to see how eps^2=0 is a good model. But the easiest to get across is that in many calculus derivations, you take dx^2 to be 0, since dx is "a very small number".