What kind of school are you referring to when you mention “elite math school”?

There are schools in various countries, e.g. Romania, which are known for producing extremely well prepared applicants. They select for these schools from the entire country and have them focus on sciences, math and computer science early on. They practice Oxbridge style interviewing/math olympiad questions to death.

This results in for instance there being proportionally many more Romanians at Oxbridge Cs/Math/Physics than you'd expect by population.

Ok, sure, but there are very few schools like that in the U.S. outside of homeschools.