> But there is something about degree-level maths and beyond that I find deeply unintuitive in a way that software development isn't.

Grok.

Over the years, I come to the conclusion that one of the stumbling blocks is the definition/concept of "application". Just like the definition of "theory" is different for a layman ("My theory is..." == "My guess is...") than from a scientist's definition ("My theory is" == "My logical framework which incorporates all of the available data is..."), so the definition of "application" is different between mathematicians and engineers.

I've noticed that math books with titles like "$HIGHER_ORDER_MATH with Applications" means "$HIGHER_ORDER_MATH with Exercises". What I'm looking for is something like "$HIGHER_ORDER_MATH with Real-World Uses".

I've known LA for decades but, like you, where would I use it in my life? The turning point for me was Andrew Ng's Deep Neural Network course.

I knew that a DNN is a program of matrix operations, but how do you get 5,000 images into a matrix? One way is to resize all the images to the same n x n size, take the first pixel of each picture and break them into their RGB components. You now have the first three rows of your input matrix. Repeat for all other pixels and voila! You have a 5,000 x n matrix that you can do linear algebra on! _That's_ an application; having me add two matrices together is an exercise.

Since that insight, I've used LA in my job in the hospitably sector with impressive success because now I know how to apply it. Math books and 3B1B show you the math. We engineers (or at least this one) need real world uses.

This is typical for many people. You love pre-college math because you have intuitive understanding, while college-level maths offer a new level of abstraction that you may not feel familiar with from the get-go.

It's perfectly okay not to learn linear algebra, by the way, especially when you don't find any incentive to do so. Otherwise, you'll find linear algebra to be one of the most intuitive tools to model so many problems.

If you do want to learn linear algebra or any other higher math, I'd strongly recommend you focus on understanding concepts intuitively first, to the point that you find many exercises in a text book straight forward. Watching 3blue1brown is a good start, but do move forward with deeper treatment. The book I find very usual is David Lay's Linear Algebra and Its Applications: https://www.amazon.com/Linear-Algebra-Its-Applications-5th/d.... Lay sets up a really intuitive geometric framework to explain the intuition of linear transformation with sufficient rigor.

Got any similar recommendations for college-level probability/stats stuff?

No. I took only a few courses on probability and mathematical stats. The books are A First Course in Probability and some some textbook on Mathematical Stats. The courses were probably not advanced enough, as I found them reasonably straightforward. I struggled a bit on what exactly is a random variable, but once that was internalized, everything else followed. I heard that advanced courses like random processes were really hard, but I'm not at that level.

Part of the challenge with linear algebra is that a lot of the basics are somewhat dry, since they serve mostly as a way to organize computation -- e.g., a system of linear equations can be expressed as Ax <= b, where A is a matrix, b is a vector, and x is a vector of variables.

Much of what makes linear algebra interesting and powerful comes from more advanced topics, especially eigenvalues. This power comes when we are not looking at a single matrix in isolation, but when we repeatedly apply a matrix. For instance, consider the equation x_t = A^t x_0. It turns out we can rewrite this an equation by diagonalizing A -- i.e., A = P^{-1} Sigma P, where Sigma is diagonal; most, but not all, matrices can be written in this form. We call s_i the "eigenvalues" of A.

Then, the equation simplifies to x_t = P^{-1} Sigma^t P x_0, or equivalently (P x_t) = Sigma^t (P x_0). This equation is dramatically simpler, since Sigma is diagonal, so if Sigma = diag(s_1, ..., s_n), then Sigma^t = diag(s_1^t, ..., s_n^t). In other words, this transformation "disentangles" the different components of A into ones that act independently. Here, the transformation x -> P x is what is called a "change of basis".

These repeated matrix applications are common in physics, where they represent how a dynamical system evolves over time. The main difference is that in physics, the system evolves continuously, but similar transformations can be applied to solve these problems.

This post really illustrates why HN needs MathJax support, and/or browsers need MathML support. Math notation is hard enough to read when you're reading the real version... when you're forced to read someone's ad-hoc "notation that I'm forced to use because I can't use real notation" it just becomes that much worse.

Luckily, Chromium is getting MathML support (finally) unless something weird happens, which (presumably) means that Chrome and Edge will inherit that support as well. Firefox already has MathML, so hopefully we'll soon be in a position where three of the most widely used browsers support MathML.

Still, it would be interesting to ask the Powers that Be if they'd be willing to implement MathJax here on HN....

The problem with Linear Algebra specifically, is that it can be viewed from many different perspectives.

The article here focuses on an "operational" perspective, how the numbers get added or multiplied together to turn into other numbers. However, Linear Algebra is also useful in geometry, and other situations.

This "intuitive guide" to linear algebra sets you up very nicely for figuring out how to add and multiply matricies together. But it doesn't give you any intuition about a rotation (aka quaternions) in 3d space, for example. A lot of math books make the mistake of trying to teach all the perspectives at the same time, instead of focusing on just one viewpoint until the student gains mastery.

A Quaternion is "just" a 4x4 matrix that represents rotation in 3-dimension space. Because you only move 3-ways rotationally (yaw, pitch, and roll), you're "underconstrained" with regards to the 4x4 matrix. Etc. etc. A lot of geometry intuition needs to be built here to really understand Quaternion... and none of that geometry is explored in the blogpost.

Which is fine. Focus is good. But when people approach Linear Algebra, its important to know that its "so useful" that there are too many ways of looking at Linear Algebra... too many different, yet equivalent, understandings of the subject.

I used to think this, my problem was that I wasn’t doing the exercises, instead just reading articles and watching videos and trying to get some kind of theoretical understanding. Everything makes a lot more sense once you’ve slogged through a bunch of repeated exercises.

100% Agreed. I've written 3D engines in shipping games and yet I can't do math for shit. I've tried watching 3blue1brown and they have pretty pictures but they don't help at me all. I feel like they're mostly appreciated by people who already understand and can't remember what it's like to not understand.

I was recently watching videos and trying to read papers on geometric algebra and getting totally and utterly lost on actually applying it.

Higher math involves a very different way of thinking from the typical, useful things people do for a living. Exactness is important. The abstractions can run very deep. It's easy to get lost in the pure side of things without really understanding how to apply it.

I think this is possibly the crux of it for me. I've certainly got well developed abstract thinking for software development, but software has always had a clear application for me, so the abstract thinking developed as a matter of course.

I've rarely found any higher math instruction which takes for the form, "so you have this specific problem X, here is how we can solve it with technique Y"[1]. But I suspect that it is because it is higher math (which presumably means 'higher order' math).

Without this, and without an inherent enjoyment of the pureness of the math, it seems somewhat esoteric for me personally. I'm not complaining, nor do I really think it should be any other way. I'm just reflecting on it really.

This also makes me think of my foray into monads: "The thing about monads is once you finally understand them you immediately lose the ability to explain what they are to others." Not saying that's the case here, just feels related.

[1] At least where I found problem X to be satisfactory. I didn't find my lecturer's problem of, "you're stood on a mountain described by this PDE, on what vector must one walk in order to stay at the same altitude" to be very applicable. I was a pretty wilful student though.

Software development never gets that abstract. Yes there are abstractions, but compared to college math, they're extremely simple. The most complex abstractions I've come across in software engineering don't hold a candle to some of the abstractions you would see in a typical undergraduate math degree. Heck, my university used Baby Rudin [0] for its introductory analysis class, which was often taken by freshmen or sophomores.

[0] https://notendur.hi.is/vae11/%C3%9Eekking/principles_of_math...

Personally I really struggle with the syntax and notation of upper-level math courses, I need a big cheat sheet of all the terms. It's like programming where we use i, j, k for loop variables instead of something more descriptive. My brain has to do one extra layer of translation between what's written and the concept being taught and then I lose focus, but intuitively I've always been fine with math concepts. When I took linear algebra in college (and did terrible) I particularly struggled with all the syntax that was introduced. Would love some tips if anyone else has a similar problem.

That is to a large extent a matter of practice. The notation can become second nature and then really helpful - you can write down regression (ordinary least squares) in two lines, basically (including problem and solution), and it all makes sense and translates fairly directly into an algorithm.

I'm currently working on a linear algebra heavy linear programming model for an optimization thesis and the general trick is, as with writing code, to improve iteration by iteration of the model. Starting with a complex mathematical model is always a bad idea, so start small and iteratively improve it. This means no use of advanced concepts unless needed. Also, nobody uses every bit of math in the everyday work. As with programming, you just need to understand the general concepts and the rest can be figured out step by step.

Try Gilbert Strang's course at MIT, it's publicly available in video format, and starts from basic principles.

I think the point where it gets confusing is where they stop showing you how to do calculations. With programming, you are always calculating something, even if you abstracted away from it.

Geometric algebra seems more practical than most subjects but even its introductions suffer from this.

That's because the way that math is taught and presented makes it extremely hard to grasp and relate to, and this makes it hard for others to find the topics within it interesting. Mathematics, in it's very essence, is incredibly beautiful. The universe - yes, everything in it and everything that composes you, is in fact a form of mathematics! What does this mean?

Well, it means that understanding it means that you understand yourself! Well, not just that, but it might eventually lead to you understanding the architecture of the universe! After that? How about God? Or is God maybe a part of the architecture?

OK, so maybe I won’t try to go into the details there since opinions on it differ. Essentially, through this language though, you can master almost anything that you want! Yes, it might seem crazy, but the things that happen around you and the things which every poet and playwright and prisoner and savior ever composed can be explained through mathematics! Through it, you can also open up entirely new and utterly interesting worlds!

Let’s say that you dropped your pen this morning. To someone with no background in physics, or mathematics, this means nothing. On the other hand, to someone like me, it means quite a bit. The slight delay and movement altered the gravitational and electromagnetic field around you and echoed on into eternity. Also, it most likely changed your days structure and composition, and shifted your life into a new line (see quantum mechanics and chaos and linear dynamics) and had a profoundly large impact on everything else around you. You may have avoided a car crash, or you might have met a person who you wouldn’t have normally encountered all because of that small change. And this small change echoed on and effected everything else in turn. And this is all very mathematical, and extremely beautiful, but most people don’t know anything about it, but it does mean something to people like me: we’re all incredibly inter-connected, and our lives are ruled by chaos. Everything that you do, and everything that you say, and all the things you see and inter-related and ohh so close, but we tend not to see it and it all has to do with not understanding the fundamental mathematics!

The above is only touching on one small aspect of it though, as it only deals with physics and chaos. There are entire branches of math which make the world incredibly interesting which have nothing to do with the fields I just mentioned! Hey, did you know that standing next to someone who might look slightly like your wife will cause you to behave in a similar manner in which you behave when she’s actually around you? Yes, neurons that fire together wire together, and it’s very mathematical underneath but yet so simple! How about the fact that E = mc squared isn’t really true? Yes, the formula has an extra term (square root of 1 minus v squared over c squared) which makes it possible for massless particles to have energy and deals with relativistic effects. Did you know that you can summarize most of modern classical physics in just a few equations? (Yup, you can find most of them here: https://www.feynmanlectures.caltech.edu/II_18.html ). How about balance? Did you know that if you were standing at arm’s length from someone and each of you had one percent more electrons than protons, the repelling force would be so incredible that the repulsion would be enough to lift a “weight” of the planet! Yup – math is full of fun surprises!

Now, modern math is sort of like a constant tease which shows you the shell of this beautiful program and this excitement, and you know the beauty is there, but it’s not easy to understand and grasp! For one, most of mathematics is filled with jargon and language that is incredibly information dense, and it looks like it’s been written by a schizophrenic C programmer who’s paranoid about losing his job, so the information tends to be lumped into these incredibly dense formulas which hide the beauty and truth, but the beauty and truth will always be there! You just need to have a bit of persistence and dedication. It also doesn’t help that most teachers tend to not make things nice. They puke out the same old standardized stuff regurgitated and taught to them, and so round and round it goes.

Hopefully though, we’ll get better at teaching it and conveying it’s structure as we learn better ways of not making things cumbersome and uninteresting to other people! Wow, I need to stop writing – sorry for the large wall of text, but I hope you get what I mean!