It took me taking a class in neural networks in my thirties to really understand matrix multiplication

I did a fairly rigorous linear algebra course in school, and it went over my head despite passing it.

Saw it explained in a single slide of Andrew Ng's ML course and everything clicked perfectly.

Where I lived math was taught like fucking shit, it was all algebra and zero context as to why it could be useful in real-life scenarios, zero abstraction such as visualization or metaphor. Everyone involved in concocting that pedagogic aberration should feel terrible about it.

The Coursera one? That's the exact course I was thinking of.

Which class?

A sibling comment to your mentioned Andrew Ng's class, and I actually had the same experience as them with the same class. You can find it by searching for Stanford CS229; the lectures are available on YouTube.

It was Andrew Ng's machine learning class on Coursera

Matrices are funny. You can encounter them as a teenager reading 3D graphics tutorials on the internet, learn that they can compactly represent scalings, rotations, and translations, and that several transformations can be conveniently “stacked” using this thing called matrix multiplication which looks like this… and that’s it, now you can use them for a cool practical purpose, but the tutorials never derive or attempt to justify why mmul looks like that, often because the author doesn’t know either!

Or you can learn about them in school and be given neither a real-world use case nor a rationale or derivation for mmul.

Or you can encounter them in college, and there the experience depends on whether it’s a good or bad kind of a linalg class. But even there – after all the painstaking definitions and lemmas and derivations – it’s easy to end up not grokking how mmul does what it does even if you grasp all the building blocks – vector spaces, bases, how matrices can represent bases and systems of linear equations and linear operations on vectors and how they’re all kinda equivalent.

It reminds me of a funny story back when I was a student.

We had a week-long group project in the first year whose theme was "a 360° pong". Our group decided that it meant the paddles had to travel in a circle around the playfield and I've decided that matrices and stacked 2D transformations were the way to go. The other students gave me blank stares, I basically said "trust me, you don't need to understand them to use them" and off we went coding.

We ended up with the most impressive pong clone out of all the groups, as nearly all of them had axis-aligned rectangular paddles going around a rectangular path, whereas ours had a stretched half-circle paddle going around a circle path always facing the center of the playfield, alongside extra features like walls and a level editor.

First class the Monday morning after, the math teacher announced that the next topic was matrices. We stared at each other in the group and grinned manically.

If anyone wants to stare at an old C codebase from 10 years ago by a bunch of first year students: https://code.google.com/archive/p/pong-norris/

And a lot of dynamics is e to a matrix.

The easiest way to describe matrix multiplication is nested function composition.

    f(x) = 2x
    g(x) = x + 5

It should hopefully be obvious that "nesting" the two isn't commutative:

    g(f(x)) = (2x) + 5 = 2x + 5
    f(g(x)) = 2(x + 5) = 2x + 10

One miraculous fact here is that no matter how many functions we stack, we only ever have two terms: x and a constant term. Thus, we can represent this 'linear system' in terms of its coefficients, as long as we agree on an order: e.g. 2x+5 might become "2,5" in our system.

You can do "multiplication" on these packs to compose them together, and even though the rules feel obtuse in abstract, they follow the logic of function composition. A matrix represents a similar function transform, only it's in 3 dimensions, and in order to handle rotation, it needs to swap around x/y/z. So an identity matrix is really saying:

   f(x) = 1x + 0y + 0z + 0
   f(y) = 0x + 1y + 0z + 0
   f(z) = 0x + 0y + 1z + 0

Who can afford to live in three dimensions nowadays?

Linear algebra in general for me was kind of one of these concepts. Aced my university linear algebra class with no idea what the heck I had learned. It didn't start to click for me until I started using it for tangible problems.

Matrix multiplication is the first example that came to mind for me too. I learned it as compositions of linear transformations (the professor “taught” it through a question on the take-home final) but it felt abstract to me and took years to become intuitive to the point where I could actually explain it from scratch.

A big one for me was realizing that a matrix times a vector returns a weighted sum of the matrix columns, with the vector's elements as weights. It's not particularly profound but it has definitely clarified my intuition around a number of matrix problems.

The "essence of linear algebra" series by 3blue1brown on YouTube does a really good job at intuitively explaining and visualizing matrix multiplication and other linear algebra topics.

Related, this always amazes me: https://news.ycombinator.com/item?id=32195262