News for hackers. Stuff that matters.
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by ajkjk on Saturday February 02, 2019 @09:39PM (#19067460)
Yes, it's incredibly faster. You do a lot of tensor algebra in a relativity class.
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by joppy on Saturday February 02, 2019 @09:56PM (#19067528)
To the point where writing one large sigma and listing indices under it at the start of each line would significantly slow you down? I found I had to do this mentally during my own relativity class, just to figure out what each expression meant.
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by frutiger on Sunday February 03, 2019 @12:20AM (#19068021)
> To the point where writing one large sigma and listing indices under it at the start of each line would significantly slow you down?

Yes (in my relativity & QFT classes). But the timesaving aspect was not that important for me. The notation enabled the intuition of “zipping” together these somewhat unwieldy mathematical objects, and that was the clincher.

Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by pishpash on Sunday February 03, 2019 @03:41AM (#19068476)
Then you're doing it wrong. The point of notation (or learning any new language) is to operate in that language, not to translate back and forth.
Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by joppy on Sunday February 03, 2019 @04:54AM (#19068689)
It's not actually a new notation though, it's literally just deleting a part of the old notation and then getting the reader to fill that part back in. While this is fine for calculations, I would rather that when equations are actually presented, that one extra piece of information is explicitly given.

I guess a similar thing that happens on the programming language side would be leaving out types when the compiler can infer them. This is great, and can make code more concise, but when reading an API I would really like the types written down explicitly, rather than having the play the part of the compiler.

Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by ajkjk on Sunday February 03, 2019 @08:31AM (#19069463)
If types are omitted then there's actual missing information from the equation. If summations are omitted, there... isn't. They add nothing (at least in relativity where every sum is over the same 4 values).

Anyway, index notation is more of a shorthand for symbolic tensor algebra computations than actual sums. The problem is that when dealing with 2, 3, and 4 index tensors, with co- and contra-variant indexes, it's quite cumbersome to notate which component of a tensor contracts with which component of another tensor, and anything you could come up with would end up looking like index notation. Rarely do you actually mean to sum anything.

Re:Einsum Is All You Need – Einstein Summation in Deep Learn... by joppy on Monday February 04, 2019 @07:25PM (#19082019)
I don't have a problem with using indices for anything, they are fine. Distinguishing upper and lower indices is even better. The exact complaint I have is the implicit sums. For example, a matrix A would be written with one upper and one lower index, which I'll just write as A[i, j]. In einstein (implicit sum) notation, the trace of the matrix is written Tr A = A[i, i]. Is this really better than saying Tr A = Sum[i] A[i, i]?