Probability by Joe Blitzstein, Harvard http://projects.iq.harvard.edu/stat110/youtube
And some linear algebra book by Strang (as others have said.) I don't remember which one I learned from, but I do remember it was excellent.
I can't recommend a good stats/probability book. People recommend Statistical Inference by Casella and Berger, although I wasn't super impressed with it.
You'll also want a good understanding of set theory and proof writing. I don't remember what book I learned from, but try to find some introduction to math thought.
This is all assuming you have a good understanding of algebra. The above mentioned books require geometry/trig, but you don't actually need that for machine learning.
Incidentally, my college calculus class also used Stewart.
My philosophy with textbooks was to buy only ones I felt I'd go back to after uni - basic references in each major field I studied - and this was reasonable for me for calculus. My only regret was shelling out for new copies because I was young and stupid and thought I needed them right now for the class, before I knew better. You can certainly pick up cheap second-hand copies per the parent post.
I also used Kreyzig too but found that to be a bit harder to learn from, albeit more in depth.
I learned from the Stewart book and it's not terrible, but I don't think that is a very good book for self learning. How to ace calculus or the calculus lifesaver, plus a book with solved examples is probably better for self study.
https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
Sites like Khan Academy, PatrickJMT, Paul's Online Math Notes, and BetterExplained are useful, too.
Coursera and edx also have offerings.
Probability and statistics will help ML as well.
I would recommend Khan Academy before any other resources, whatever your level. It has online exercises with auto-correction, and a mastering system that uses spaced repetition. This will ensure you'll actually assimilate the material. After you can use another resource as a reference.
http://statweb.stanford.edu/~tibs/ElemStatLearn/download.htm...
https://www.amazon.com/Elements-Statistical-Learning-Predict...
vs
https://www.amazon.com/Introduction-Statistical-Learning-App...
The former seems to assume prior knowledge and the latter seems more beginner friendly, except it uses R (in case anyone wants the distinction).
I would like to add www.codingthe matrix.com for any developers, which has a MOOC as well (Coursera), as a complement to Strang.
A lot of the material in this book makes much more sense having had experience coding machine learning algorithms.
I don't want courses that teach math by coding but actual math courses. A lot of these online ML courses treat math as a black box so you never fully understand what's going on & a lot of math based ML courses are too advanced.
So I'd rather focus on understanding the math basics before diving into ML. So what do you guys recommend?