I was running into the same issue. I wanted to get into deeeplearning but my math skills had atrophied. go check out mathacademy.com. its no where near the level of time investment that going back to college is and you will learn a lot!