Math, properly done, is rigorous formal thinking. And it lets us think things we normally couldn't. Nobody can visualize a 100 dimensional object, but a mathematician can easily work with one.
And as programmers we work with mathematical objects called state spaces, that have vastly more than 100 dimensions.
That said one can easily be a competent programmer without having much formal mathematical knowledge much like one can easily be a competent ball player without knowing the differential calculus. However, just as modern ball players improve their games with computer aided mathematical analysis of their swings and so on, a programmer can improve the quality of his output by mathematical analysis, in particular via the use of the predicate calculus and its, in my opinion, most useful application of loop analysis.
I agree with your gist, there are lots of things where studying that thing is virtuous beyond its direct application. But also, I’d contend that thought is the subject of mathematics and not just a virtuous side-effect.