News for hackers. Stuff that matters.

A comment on Einsum Is All You Need – Einstein Summation in Deep Learning (2018).

Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by Xcelerate on Saturday February 02, 2019 @08:47PM (#19067247)
I remember when I was learning matrix calculus and realized at some point that it was much simpler to convert everything to index notation, perform all operations, then convert everything back to standard notation at the end. It became almost comically simple, because you're "just" working with labeled scalars at that point. To be fair, it's convenient to memorize some of the more commonly used expressions (like ∂tr(AB)/∂B) rather than rederive them from scratch.
[ Parent | on HN ]
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by krastanov on Sunday February 03, 2019 @12:59AM (#19068114)
Roger Penrose (the famous mathematician) has been saying the same thing for a couple of decades (going even further than that, check out Penrose diagrams).
[ Parent | on HN ]
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by jesuslop on Sunday February 03, 2019 @03:39AM (#19068470)
And Penrose diagrams (tensor networks) became a prominent example of String Diagram in the monoidal category of vector spaces over a field.
[ Parent | on HN ]
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by cdavid on Sunday February 03, 2019 @06:45AM (#19069089)
I am interested: do you have a reference that shows this through examples ? In my grad school years, I was always dissatisfied by matrix calculus notations
[ Parent | on HN ]
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by cdavid on Sunday February 03, 2019 @06:44AM (#19069086)
Interesting, I almost always got the exact opposite experience. How do you calculate non trivial matrix derivatives using index notation ?
[ Parent | on HN ]
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by mcabbott on Sunday February 03, 2019 @07:25AM (#19069241)
You insert kronecker deltas: ∂M_ab / ∂M_cd = δ_ac δ_bd , and a,b here remain contracted with whatever M was originally contracted with. Then you simplify, δ_ac Z_xya = Z_xyc and so on.
[ Parent | on HN ]