You don't need any prior math knowledge to attend college. The math they teach at high school only serves to develop a bit of mathematical maturity, but it's quite different. The abstraction level is much lower, there is little emphasis on proofs, and too much emphasis on elementary algorithms to solve some oversimplified problems.

Furthermore, unless you have followed some relatively uncommon high school program that teaches logic and discrete mathematics, the actual overlap between high school and first year CS math is almost empty.

I think a major part of excelling at mathematics is to understand that you must go through material very slowly. Mathematics is not literature. Taking 1 hour of good effort to go through 1 page is far from unusual if you want to truly conquer concepts by understanding definitions, developing good intuitions and proving all major results.