Tensor considered harmful

Re:Tensor considered harmful by The_suffocated on Friday January 04, 2019 @12:24PM (#18825778)

I ain't saying that deep learning is unrelated to tensor algebra. E.g. low-rank approximation of matrix can be reformulated as a tensor problem. Apparently, the following discussion in CrossValidated suggests that some deep learning researchers do try to do something clever using true tensors:

https://stats.stackexchange.com/questions/198061/why-the-sud...

But my point is that the current "tensor" nomenclature in deep learning software has little to do with tensors in physics or mathematics. The name is just confusing if not misleading.

[ Parent | on HN ]

Re:Tensor considered harmful by jhanschoo on Saturday January 05, 2019 @12:52AM (#18830175)

What uses of MDArrays in deep learning don't correspond to tensor manipulations? Many aspects of deep learning theory are discussed in a coordinate-free manner as tensors or multilinear maps, with the MDArray simply referring to the coordinate representation. The operations on the MDArray then refer to a change in representation of the underlying linear-algebraic object, or a discretized version of a linear-algebraic operation on the underlying object. Hence I find the naming entirely appropriate.

[ Parent | on HN ]

Re:Tensor considered harmful by The_suffocated on Saturday January 05, 2019 @05:01AM (#18830844)

What uses of MDArrays in deep learning don't correspond to tensor manipulations?

That's news to me. All genuinely tensor-related operations in machine learning that I was aware of were about dimension reduction (tensor decompositions or low-rank tensor approximations). Maybe things have changed and I stand corrected.

[ Parent | on HN ]

Re:Tensor considered harmful by nabla9 on Saturday January 05, 2019 @09:00AM (#18831661)

Remember that NNs do inference forward but learning happens backwards.

[ Parent | on HN ]