Re:Fully Automatic Differentiation for Tensor Expressions
by melvinzzz on Tuesday May 22, 2018 @12:53PM
(#17127369)
Here the way autodiff for a simple 1-D concatenation is computed. First we write the concat itself:
Note, that SA and SB are the sizes of the two tensors, and due to the way that edge case handling works, either A or B is out of bounds and thus equivalent to zero for each index i of O. Now in the case of +/+ accumulation/combination, the autodiff writes the derivative of each input as the sum over only the output (rather than multiplied by the other inputs as in the +/* case). This gives us:
A further reduce/defract slightly changes this to:
which basically extracts the proper portion of the derivative of dO into dA and dB respectively.
[ Parent
| on HN ]
There are no comments on this comment yet.