Matrix Calculus for Deep Learning

Re:Matrix Calculus for Deep Learning by jules on Tuesday January 30, 2018 @06:24PM (#16270049)

The notation is very simple: you write tensor expressions with indices, and repeated indices are implicitly summed over. For example, matrix multiply:

   A_ij = B_ik C_kj

Differentiating this with respect to variable l:

   ∂_l A_ij = ∂_l (B_ik C_kj) = (∂_l B_ik) C_kj + B_ik (∂_l C_kj)

By writing out indices you can just use the rules for scalar derivatives.

[ Parent | on HN ]

Re:Matrix Calculus for Deep Learning by jph00 on Tuesday January 30, 2018 @08:05PM (#16270501)

Oh in that case I'm just confused - that's what I know as Einstein notation. Modern physics papers seemed to use much more complex notation, but I probably just misunderstood.

If we're talking about Einstein notation, then I'm a fan - `np.einsum()` is often a great way to create fast tensor computations with minimal code.

[ Parent | on HN ]