News for hackers. Stuff that matters.
Re:Tensor Considered Harmful Part 2 by thanatropism on Monday January 14, 2019 @09:22AM (#18903102)
We've got a winner. Tensors "do" something; they're functions.

In the rank-1 and rank-2 case these functions can be represented as objects familiar to linear algebra. 3-tensors are not the same as stacked two-tensors, however.

I wonder if "neural networks as ODEs" will catch on. I could use some of my masters work and go get a PhD.

Re:Tensor Considered Harmful Part 2 by joshuamorton on Monday January 14, 2019 @12:47PM (#18904766)
>We've got a winner. Tensors "do" something; they're functions.

And a neural network is just a stack of linear transforms (the "tensors") and nonlinear transforms (the activations). So in a lot of cases, these "tensors" do do something. Granted the abstraction for storing the active tensors is also a great abstraction for storing high-dimensional data (like a 128x128x4x60 2-second video).

>neural networks as ODEs

You're a few months late :P [0]

[0]: https://arxiv.org/abs/1806.07366

Re:Tensor Considered Harmful Part 2 by twtw on Monday January 14, 2019 @10:47AM (#18903824)
Is it accurate to say that rank-n tensor is to n-D array of numbers as linear transformation is to matrix?
Re:Tensor Considered Harmful Part 2 by krastanov on Monday January 14, 2019 @11:16AM (#18904061)
For a mathematician or a physicist, yes. But the word ended up having new, confusing meanings.
Re:Tensor Considered Harmful Part 2 by nabla9 on Tuesday January 15, 2019 @08:16AM (#18910892)
It's the same for ML perspective as well.
Re:Tensor Considered Harmful Part 2 by Koshkin on Monday January 14, 2019 @07:14PM (#18907837)
> they're functions

Well, in math a lot of things we normally do not think of (or use) as a function can still be considered as such. For example, a vector "is a function." Why? Because it does something (to a covector, or a linear form - which, in turn, is a function, etc. etc.) Even a number "is a function" by a similar reasoning. Objects in math are very flexible conceptually.

Re:Tensor Considered Harmful Part 2 by thanatropism on Monday January 14, 2019 @10:09PM (#18908589)
Ehhh.

Take for simplicity a constant real number. A constant is not a function but there's a function from the reals to a space of real valued functions. Likewise every vector can be converted into a cove tot, but...

You can say these function-valued functions are bijective so you can work on either side of the fence. Your can also work with equivalence classes if you don't quite have a bijection. But a lot of mathematics is about being specific about what we want, even if we're lazy in writing later and smush related concepts together in the same symbol.