Frankly, compared to many other disciplines, that's pretty light. Single variable calculus, linear algebra and probability were all required in my undergrad CS curriculum. Multivariable calculus was the only extra course. And if you're in most engineering disciplines, all of this is required.
I'm guessing the difference for CS folks is that they rarely use this stuff in the curriculum, whereas in most engineering programs, you'll use calculus day and night.[0] So for someone like me (engineering background), it was easy to dive into even years out of school as I'd not forgotten a lot of math.
Now of course, if you want to get deep into ML, there's a lot more math than that. However, most successful people using ML do not need to know that math.
And compared to other disciplines like control theory, communications theory, etc, the prerequisites for ML are a lot lighter.
[0] Only in school. Almost never in industry.
If you’re someone who enjoys math you might do it for its own sake but it’s difficult for me to imagine someone making serious progress in this direction without either a passion for the subject, an already relatively strong background, or being in a university.