One possible idea is to get started learning math through physics. The reason is that physics is more applied than pure theoretical math, but the typical way of learning it makes use of similar math principles. Perhaps seeing this stuff in use will help pull you along when working through the pure math subjects.
A second issue (note the time period) was that I saw what the CS students were learning, and I also had a summer internship in a computer facility, and I formed the impression that over the long haul, a programming job would be rather dull. The people who were doing things with computers that seemed "cool" to me, were in the math and physics departments. That included the use of microcomputers rather than mainframes.
I've certainly done a lot of programming throughout my career, but have not been employed as a programmer per se. Most physicists program, because it's our most efficient and flexible problem solving tool.
I could probably claim to be a "data scientist" right now, if push came to shove, but am in fact working through some training on that and machine learning, to fill in gaps and make me conversant in the terminology. It could propel my next career move, or at least serve as a bit of career insurance.
What kind of physics? Big banks on wall st hire(atleast used to) pure physics phds and train them to do quantitative analysis for finance
Thanks in advance