Einsum Is All You Need – Einstein Summation in Deep Learning (2018)

Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by mcabbott on Sunday February 03, 2019 @05:10AM (#19068750)

If you have just one term, then leaving out the summation sign saves little. But in larger expressions, the rule applies sums per term, and then the savings in clutter can be quite large. For example, here v_i is not summed, and the term with z_b has two sums:

A_ia x_a + v_i + x_a y_a ( w_i + B_ib z_b )

You should not need to check the LHS, x_a y_a should always mean the dot product of these two. At least this is the convention among the heavy addicts; among lighter users you will sometimes find other setups.

[ Parent | on HN ]

Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by joppy on Monday February 04, 2019 @07:29PM (#19082041)

I still feel that the whole expression could be clarified quite easily be writing a \Sum_{a, b} at the front?

[ Parent | on HN ]

Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by mcabbott on Tuesday February 05, 2019 @04:12AM (#19083897)

But that would sum everything, not just the parts which need it. You'd actually have to write this:

\sum_a(A_ia x_a) + v_i + \sum_a(x_a y_a) ( w_i + \sum_b(B_ib z_b) )

My example is actually still easier without indices. (Although you lose some information, like the fact that x & y are vectors in the indexed by a,b.) Here it's clear that Ax and x⋅y each involve a sum, which never includes v:

Ax + v + (x⋅y)(w + Bz)

[ Parent | on HN ]