News for hackers. Stuff that matters.
Re:Vectorization: Introduction by amelius on Thursday June 01, 2023 @06:05PM (#36158050)
We really need to name tensors differently. "Array" would be a more fitting name.

Tensors are really different mathematical objects with a far more rich structure than those used in the Deep Learning context.

https://en.wikipedia.org/wiki/Tensor

Re:Vectorization: Introduction by sva_ on Thursday June 01, 2023 @06:28PM (#36158297)
Names can have different meanings based on context, even within pure mathematics. I think a mathematician would have little trouble discerning which kind of tensor is meant, as this sort of reusing existing terms is pretty common.

Just calling it an array might be underselling it, at this point. Perhaps a tensor in the context of CompSci is just a particular type of array with certain expected properties.

Re:Vectorization: Introduction by rsfern on Thursday June 01, 2023 @09:29PM (#36159756)
> Just calling it an array might be underselling it, at this point

I like the NDArray terminology used by numpy. I think it gets the point across more clearly than “tensor”, and conceptually they’re pretty much equivalent

Re:Vectorization: Introduction by itishappy on Thursday June 01, 2023 @06:11PM (#36158105)
Re:Vectorization: Introduction by antognini on Friday June 02, 2023 @01:05AM (#36161244)
As a transplant to the ML world who migrated from physics I do find it funny that people in ML will insist that gradients are vectors. Once you get to general relativity your physics professors will beat into you the fact that gradients are not vectors. (They are one-forms.) But in the ML context it's a distinction without a difference, unless you're in a really specialized corner of the field like information geometry.
Re:Vectorization: Introduction by ReleaseCandidat on Friday June 02, 2023 @04:09AM (#36162481)
> Once you get to general relativity your physics professors will beat into you the fact that gradients are not vectors.

In linear algebra you should learn this as soon as you hear about it. Is the english Wikipedia page correct in that you always use nabla for the gradient? https://en.wikipedia.org/wiki/Gradient#Generalizations

Because `grad f = ∇f` holds in scalar fields f only (not in vector or tensor fields (tensors that aren't scalars)).

Re:Vectorization: Introduction by antognini on Friday June 02, 2023 @01:07PM (#36167839)
Even if you are taking the gradient of a scalar field, the resulting object is not a vector. The difference is that if you apply some transformation to your coordinate system a gradient transforms differently from a normal vector.

In the ML context it generally doesn't matter because you don't do these transformations (and more generally there is no metric for your space). So you can just treat a gradient as an array of numbers. But in a physical context the distinction starts to matter, at least on a manifold with curvature.

The one place it does matter in ML (that I'm aware of) is information geometry, where you try to do a transformation from the normal coordinate space to a more "natural" coordinate system that is based on the Fisher information, and this introduces curvature into model.

Re:Vectorization: Introduction by opportune on Thursday June 01, 2023 @06:17PM (#36158170)
You could apply that argument to “isomorphic” functions, c++ “vectors” or compute “vectorizarion”, “integer” types… even though the semantics aren’t exactly the same between the computing and math terms, they’re related enough that they do help with understanding to those coming from a math background (which to be fair is probably less and less programmers over time).
Re:Vectorization: Introduction by myownpetard on Thursday June 01, 2023 @06:30PM (#36158313)
Isomorphic javascript is one of the most egregious.
Re:Vectorization: Introduction by nologic01 on Thursday June 01, 2023 @06:20PM (#36158214)
I think that naming battle is lost.

Would indeed be cool to have true tensor processing units :-)

PKD level of cool as you could argue those chips directly process spacetime chunks

Re:Vectorization: Introduction by sroussey on Thursday June 01, 2023 @07:35PM (#36158913)
Tensor processing units sounds far more cutting edge than number array processing units.