> Suggesting that more poor children than wealthy children will impoverish society in more than just a statistical sense implies there's something immeasurably special about the people who become wealthy and famous, as if the mere act of breeding with them will produce superior humans.

Every trait we've managed to come up with an objective measure for is heritable, to a lesser or greater (but usually greater) extent. "Of course intelligence isn't heritable" - but somehow score on every test we can think of for it is. Same for creativity. Same for personality traits.

Children largely resemble their genetic parents; we don't have to like it but it's still reality. If the people who contribute the most to society, who help out their fellow humans the most (and I'm not saying that's the rich - but it is largely, as grandparent said, the successful), have fewer children than those who don't, we should be worried.