I don't find CS-level mathematics difficult, but still failed it twice during first semester because at that time it's the most boring, and pointless thing you can learn when trying to become a proffessional programmer.

I picked everything up by myself later after learning computer graphics and AI, since that's when, for me, algebra and calculus became something more than a grade. Derivatives "clicked" for me because i needed to know how to calculate a root value using Newton's method.

I do lack fundamentals, and i do suffer because of that, but over the years i came to the conclusion that i won't understand something if i don't care, and that's just how things are. The education system tends to front-load theory, and i can't learn for the sake of learning. Maybe it's the same for you.