To expand on that, read Paul Graham's essay "Beating the Averages."
http://www.paulgraham.com/avg.html
You're stating, unequivocally, that math doesn't directly affect the quality of your code. In fact, in seems to me[0] that the vast majority of people who repeatedly declare that math is useless or of minimal use in coding are themselves not qualified in math. To them it's just all abstract nonsense that they don't need.
Contrast that with people who have got deep qualifications in math who say, repeatedly, that it does affect the quality of their code, and it does help.
What do you make of that?
If you don't have any background in math then you won't have the knowledge and experience to see when it will be useful.
[0] I'd love to see a proper survey/study of this.
This. These are things that will directly affect the quality of your code.