I just flipped through that course. It covers just vectors, matrices, probability, and derivatives. A better (and faster) resource is to use chapters 2-5 of Deep Learning by Goodfellow et al, but the mathematical detail in that book may be offputting for beginners.

I agree with your comment. The AWS course recommends that you should skip the class if you either minored or majored in math.