I don't believe for a second that understanding of math is at all correlated to efficiently storing and manipulating raw numbers in your brain.

If anything, it can help you survive school where inadequate teachers try to make you believe the direct opposite.