Math is a very specific ability and interest, requiring not just high general intelligence but some additional elusive factor (also likely heritable, as anybody observing "academic dynasties" would note). There is research on this [1] [2], but not nearly enough to say much more.
Anyway, I hope I explained my position on this. It's not the specific gene variants or mechanisms that matter, but basic threshold effects over polygenically heritable traits, and hard diminishing returns on teaching someone who does not meet the talent requirement.
1. https://journals.plos.org/plosbiology/article?id=10.1371/jou... 2. https://pmc.ncbi.nlm.nih.gov/articles/PMC5290743/
Beyond basic arithmetic, the utility of math is not nearly as obvious or widely applicable. It feels much more detached and abstract and this is made worse by popular teaching methods (which frequently lack hands-on examples of the math in question in practical application). It’s only natural that many children, even those who are capable, don’t take interest.
In other words, nothing about reading is natural, and nothing about what you're saying is a "fact of nature."