Re:Matrix Calculus for Deep Learning
by SoerenL on Tuesday January 30, 2018 @05:14PM
(#16269600)
But how do you compute the derivative of x'Ax in Mathematica (x being a vector and A being a matrix)? What you have pointed out is only scalar derivatives, if I am not mistaken here.
[ Parent
| on HN ]
Re:Matrix Calculus for Deep Learning
by improbable22 on Tuesday January 30, 2018 @05:27PM
(#16269668)
Like this, perhaps?
A = {{1,2},{3,4}}
vec = {x^2, x^3}
D[vec.A.vec, x]
Or perhaps like this, again the table of derivatives:
xvec = {x1,x2}
Table[ D[xvec.A.xvec,x] ,{x,xvec}]
(all untested... one typo caught...)
Mathematica can definitely compute the derivatives if you fix the size of the matrix. This isn't very useful if you're trying to compute the derivative of an expression with arbitrary sized matrices.
Re:Matrix Calculus for Deep Learning
by improbable22 on Tuesday January 30, 2018 @07:48PM
(#16270437)
You can do general matrices too, what do you have in mind?
aa[x_] = {{1, 2}, {3, 4}} x
bb[x_] = {x^2, x^3}
D[ a[x].b[x] , x, x] (* for any suitable tensors *)
% /. {a -> aa, b -> bb}