News for hackers. Stuff that matters.
Re:Tensor Considered Harmful Part 2 by contravariant on Monday January 14, 2019 @03:58AM (#18901661)
To the best of my knowledge the operations in deep learning are very much tensor operations. In most cases they seem to boil down to matrix multiplicaiton on one of the indices, but it's still kind of useful to use tensor notation (especially using the Einstein summation convention) since IMHO T_abc M_ax = S_xbc is easier to read than the corresponding sum, or the corresponding pseudocode.

Not sure if the fact that tensors have slightly more structure than mere arrays w.r.t. e.g. a change of basis is that useful, but it could be useful in proofs, especially if a FFT is involved somewhere.

Re:Tensor Considered Harmful Part 2 by mcabbott on Monday January 14, 2019 @06:43AM (#18902304)
Some operations are quite tensor-ish. For instance the re-shaping which einops writes like this

    Rearrange('b c h w -> b (c h w)')
is literally a tensor product of the vector spaces indexed by c,h,w. The same space that np.kron maps into (possibly modulo the order of indices) rather than np.hstack (of row vectors) which maps to the direct sum, ⊕ not ⊗.