The author is perfectly clear in the first sentence that the piece's focus is about the usefulness of tensors in a physics context.
Technically, that’s a tensor field which is a section of the tensor bundle. Similarly, a vector field is a section of the tangent bundle (the collection of all the tangent spaces of the points on the manifold). A vector field is just a choice of a tangent vector for each point from that point’s tangent space.
In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick.
If you think developers are guilty of circular logic, check out some of the math pages on Wikipedia. You can get lost in moments.
Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?
That’s a pretty good bar and I wonder if we could just cut to that chase earlier. But I also believe that people need enough math to see when they’re being cheated, and I feel like you could just tell middle schoolers that and they would pay attention. Maybe even primary school.
You told Billy he could have three apples, and now there are two left. Did Billy take more apples than he should have?
It’s always how do you share your cookies fairly with your friends and if they’re my cookies why do I have to share them at all? Screw “fairly” I’m keeping the extras at least. That sort of sharing is a socially advanced concept they don’t entirely get just yet.
Actually being able to do stuff with Bayes law by hand is going to be not only hard to teach, but probably impossible to remember for those of us who don't actually do math in real life. People forget stuff after a few months or years.
I highly doubt the average person is interested in checking the math on a science paper, so if you want the general public to understand statistics you... have to show us all a reason to, and also teach us all of the related skills needed to make it useful. Or else.... we will all just forget, even with the best teacher in the world.
Most of us aren't doing random game engines as a hobby project or testing things on bacteria cultures.
Maybe they should teach it in context of how to understand a scientific paper, since that's one of the more relevant things for non-pros. If you just teach statistics alone people will say
"Ok, now I know that it's easy to lie to yourself if you don't use any numbers but I don't have collections of large numbers of data points in my life to actually analyze"
If you have a way to fix this, you would be set for life, going around playing a sort of corduroy jacketed Robin Hood, keeping the rich from stealing from the poor.
Realistically, it is (somewhat) fixable in small contexts. I've worked (and continue to work) on teams that are somewhat decent at risks and probabilities, but it's definitely an exceptional experience.
I don't know how to widely teach that. But I'm not yet ready to give up and say "it can't be taught", because those folks on my team are the counterexamples.
In upper-level undergraduate math, I made a game of seeing how many pages I would go before seeing 7 printed anywhere. It was usually 10 pages, if I included the page numbers.
Here's an example:
https://en.wikipedia.org/wiki/Homology_(mathematics)
It has motivation, examples, and even actual numbers (though they're really just 0 and 1. most of the time). In my opinion, it's very good and clear exposition, for an encyclopedic article. However, I strongly suspect that people without enough mathematical knowledge (and "enough" in this case is something in the neighborhood of "enough to obtain an undergraduate degree in Mathematics") will simply not get anything about it beyond "it's about number of holes" (and that's not even remotely close to the whole picture: homology theories are important and useful in context of things with no "holes" to speak of). If you think otherwise, but not know what a quotient group is, you're just fooling yourself.
This is something I observe on HN a lot: people don't understand advanced mathematics, and are dumbfounded by the fact, trying to blame weird notation mathematicians insist on, or lack of motivation/examples/pictures etc. I never see people here do the same with advanced physics ("if the Standard Model is so standard, why can't they briefly and clearly describe what it is" is not something I ever see), molecular biology, or material science. People seem to know their limits and understand that really grokking these fields requires many years of deep study.
I think it's because many people on HN have good experience learning mathematics at school: it was something they always grasped really easily, and were easily able to figure out how to calculate derivatives, integrals, get matrices into normal forms etc. I don't want to rain on anyone's parade, because these things are still relatively difficult, and it does require more intellectual ability and effort that probably 3/4ths of the population aren't capable of. However, relative to advanced mathematics, undergraduate calculus is really rather trivial stuff.
Point is, if you don't understand modern advanced mathematics, you shouldn't get any more disappointed than you are about not being able to play violin. These things just don't come easy.
Eg "Numbers (formal) are those objects which behave like numbers (informal)."
It is still a work in progress, but does it help address some of the problems you see in learning mathematics? Any feedback is greatly appreciated. Thanks.
TensorFlow "tensor"(and most other use of "tensor" in programmer jargon) is not a tensor at all, it's just a multidimensional array.
Matrices (or linear transformations in general) are important examples of tensors. There's a nice adjunction between tensor spaces A(x)B and the space of linear transformations B=>C given by:
Hom(A(x)B, C) = Hom(A, B=>C)
In the case of Tensorflow I think they do actually still talk about linear transformations of some kind so it's perfectly fine to call them tensors.
Tensors are introduced by physicists to ensure various physical quantities (which involve coordinates and their derivatives) do not depend on the arbitrarily chosen coordinate system. This is ensured through the transformation properties of tensors.
The name tensor itself comes from the theory of elasticity, Cauchy stress tensor, which BTW is uniform in many practical cases, and obeys the following tensor transformation rule:
https://en.wikipedia.org/wiki/Cauchy_stress_tensor#Transform...
like any other (contravariant) tensor must.
Matrices are not examples of tensors. Matrices can be used for representation of tensors, in which case tensor product becomes Kronecker product, but matrices in general don't have to represent tensors. You can put anything, including your favorite colors or a list of random numbers, in a matrix, and it won't be a tensor in general, not unless it must transform like a tensor under coordinate system changes.
Similarly, TensorFlow "tensor" is just a multidimensional data array, with no transformation rules enforced on it, and therefore is not a tensor.
it’s like some people invented a new word and won’t tell you what it actually means in sufficient detail to differentiate it from all the other words you know. so you keep using it with others in the hopes that contextual information will finally make it clear. one day.
Thankfully, there is a great historical example of this. The electric field vector \vec{E} = (E_x, E_y, E_z), is not a tensor. It doesn't obey the tensor-transformation law. Similarly, the magnetic field vector is not a tensor. These are matrices, but not tensors.
As you know the Electromagnetic tensor [1] is the tensor that correctly transforms under coordinate transformation, and hence allows different observers to agree with each other.
ps - one thing that always annoyed me was the limitation of linearity in so many of these models (which i totally understand why, but still). all the interesting real-world stuff happens non-linearly...
In the words of Dirac, "I consider that I understand an equation when I can predict the properties of its solutions, without actually solving it." This usually only works if your equations are linear.
[0]: but at this point, not really. even in grad school, i only did linear modeling, and relatively rudimentary ones, at that.
But not any multidimensional data is a valid tensor.
[ 1 2 ]
[ 3 4 ]
FYI, matrices transform between vectors within a vector space.
You bring up a good point though, if this were meant to be a transformation, then we're talking about modules (Z/2^8Z being the underlying ring) and not vector spaces, which is fine. I was needlessly narrow when I said "vector spaces" earlier.
Why do you mention things like data structures, enums and labels? This is math, not C++.
Anyway, the point is, any arbitrary 2D arrangement of numbers can be a valid matrix, whatever those numbers may represent.
I'm genuinely claiming they aren't numbers due to the explicit context. The entries are colors with numerals as labels.
Now, you've said something slightly different where I think we can agree. If the entries are from a set closed under a notion of multiplication and addition, then it represents a matrix even if it isn't being used in that way. But I still say that if you're merely using the array as a place to keep some data then I won't be using the word matrix, I'll just call it an array.
Anyway. I started this by saying it was interesting that we use the word matrix slightly differently. I still think that's interesting and think it's totally fine if you want to call an array a matrix even if the entries don't come from something where a transformation makes sense. I wouldn't "correct" someone's usage. I just think it's interesting how we use it different. Anyway, I feel like you just think I'm dumb, so I'm ending this here. You might not, but it's hard to read the vibe via text. If you have something to add you think will get me to change my word choices, feel free to respond and I'll read it, but I'm not replying on this thread anymore.
Square ones do, but m x n ones represent linear maps from an n-dimensional to m-dimensional vector space (over the/a field containing the elements of the matrix).
Sure there are: Any basis of the underlying vector space(s) induces a basis of the tensor space. Components respective to some basis are coordinates. You can then investigate what happens to the induced basis (or rather, the respective components) under a basis transformation of the underlying vector space(s), which is where the "physicist's" definition of tensors originates.
A change of coordinates does indeed induce a change of basis, but a change of basis isn't really a change of coordinates. And strictly speaking some vector spaces don't really have an obvious basis (without invoking choice), so having a basis be a prerequisite for the definition is not ideal.
The whole requirement that a tensor is 'something that transforms like [...] under a coordinate transformation' is just how physicists have chosen to phrase that a vector bundle is only well defined if it's definition isn't dependent on some arbitrary choice of coordinates. In my opinion this requirement is more easily apparent in the mathematical definition where there is no choice of coordinates in the first place, rather than the physicists way of working with some choice of coordinates and checking how things transform.
However, physicists get introduced to tensors far earlier than any excursions into differential geometry when discussing rigid bodies.
Wald's approach in General Relativity is much better - he treats Tensors as a multilinear map from vectors and dual vectors to scalars.
He then derives the underlying coordinate transformaton rules, for the vector spaces used in differential geometry. But
Most of the article tries to provide some intuition behind why multilinear maps, which sound like a fairly abstract concept, might be relevant in physics. The key link being the importance of coordinate invariance.
I didn’t go into deriving the coordinate transforms from the multilinear map definition as I didn’t feel that it’d provide much better intuition, but I did mention the equivalence near the end.
Yeah, the idea that there are pre-existing things that we're trying to describe is somewhat weird to me when we're trying to come up with a definition of a tensor. The whole point of mathematics is that you come up with the definitions and theorems fall out.
In particular, this comment is funny and speaks to some difference in how I and the author view what we're doing when defining a tensor:
> But why that specific transformation law - why must tensors transform in that way in order to preserve whatever object the tensor represents?
Because we defined it like that! When you make the definition "a tensor is a thing that follows X laws", you don't get to ask why, you just defined it!
Just a funny bit of phrasing, I get what is meant :)
That's just how it's presented in textbooks. It's obviously not math is actually done.
>[the explanation in the OP] will make sense to physicist, but no sense to most everyone else, including mathematicians
And then went on to describe tensors in a way that is unfriendly to non mathematicians by saying
> tensors are a sort of universal objects representing multi linear maps: bilinear maps V x W -> R correspond canonically one-to-one to regular linear maps V (x) W -> R, where V (x) W is a vector space called tensor product of V and W, and tensors are simply vectors in this tensor product space.
For example the stress and strain calculations which are used for calculating Deformation (Say if you were rolling a sheet of steel in a mill) makes use of tensors and also something called an "Invariant" I assume this also comes from Physics/Mathematics world.
Even as a physicist I found it highly confusing when I got told in physics classes that a tensor is "just a thing (or object) that behaves like so under coordinate transformation". Like, what do you mean by "thing"? I have no intuition to this yet, I need it concise definitions! Fortunately I took a differential geometry class at the same time, which was really helpful.
For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is some object out there, and we’re trying to understand what it is in terms of how it behaves. It also uses the summation notation which is rather foreign to non-physicist mathematicians. Then, when it finally reaches the point where it is all related to tensors in TensorFlow sense, we find that there is no reference made to the transformation formula, purportedly so crucial to understanding tensors. How comes?
The solution here is quite simple: what author (and physicists) call tensors is not what TensorFlow (and mathematicians) call tensors. Instead, author describes what mathematicians call “a tensor bundle”, which is a correspondence that assigns each point of space a unique tensor. That’s where the transformation rule comes from: if we describe this mapping in terms of some coordinate system (as physicist universally do), the transformation rule tells you how to this description changes in terms of change of the coordinates. This setup, of course, has little to do with TensorFlow, because there is no space that its tensors are attached to, they are just standalone entities.
So what are the mathematician’s (and TensorFlow) tensors? They’re actually basically what the author says, after very confusing and irrelevant introduction talking about change of coordinates of underlying space — irrelevant, because TensorFlow tensors are not attached as a bundle to some space (manifold) as they are on physics, so no change of space coordinates ever happens. Roughly, tensors are a sort of universal objects representing multi linear maps: bilinear maps V x W -> R correspond canonically one-to-one to regular linear maps V (x) W -> R, where V (x) W is a vector space called tensor product of V and W, and tensors are simply vectors in this tensor product space.
Basically, the idea is to replace weird multi linear objects with normal linear objects (vectors), that we know how to deal with, using matrix multiplication and stuff. That’s all there is to it.