You could be affected by any number of things (wrong university, wrong goals, wrong order of operations, in a hurry). Please don't be offended, I just mean that you might leverage your resources better and find classes you care about (I ended up double majoring in CS and Economics, but I didn't care about econ until I tried it).
Here is a more plain and TLDR response. Pretend it's a new investment and write down/spell out what you are putting in and getting out. If it doesn't add up, don't make the investment.
Finally, not caring is a problem. For some arguing in favor of degrees, this is a big part of their point. Why don't you care about those classes? Do you think you'll only ever get cool and interesting projects at work? An employer needs to know you'll spend two weeks finding some ridiculous bug that doesn't even affect anything if that is what they assign. Every third Sunday, the system crashes. Fix it. While your peers do greenfield work. It means sometimes putting effort into things you aren't interested in as part of the bigger picture.
The more interesting work goes to programmers who can communicate. 90% of programmers can get the job done; I want someone who can articulate what he's built, how it can be improved, when it will be ready, what needs to be changed, what we're doing wrong, etc.
That's what all those classes you don't care about are for: giving you perspective outside heads-down coding that can be outsourced for $50/day.
You'd be amazed at how many people I have interviewed that simply either can't, or won't talk (from the interviewer's perspective they are the same thing!).
Again: most halfway decent programmers are adequate. The great ones are great not because they can write code, but because they can explain how that code works to someone else and, vice versa, they can understand someone who is trying to explain why the code doesn't do what they need. That is worth paying for.
The other day I finally cut a guy off and said, "Yes, sure, you aren't good with terms and explanations. How do you have technical discussions with other developers? What do you do in code reviews?". Deer meet Headlights.
I also realize your point is beyond technical communication, and you are right on that too.
Other classes folded back into CS or working for a business as well. For example, my psychology course was helpful in a number of areas, including as preparation for an AI elective I took.
We also have a requirement for attending a philosophy class about ethics in computing or something.
Somewhere in there I took a few actual EE courses.
Guess which ones had the most impact on my career? Hint: it ain't the EE stuff (although I worked as an EE for 6 years after graduation).
Computer science is only focused on logic and algorithms with state.
It's not really math.
But in the end I DON'T mind math classes, I like math classes. I mind stupid English, Lit, and Phil, classes.
I'll probably go for a math minor. I like it. But I'm not going to for a English minor.
Right now my in major GPA is above a 3.2 while my cumulative GPA is hovering around a 2.9; I just don't care about non-cs classes.
This would be find if being at a university had any noticeable benefits, but it only has detractors.