Consider the cottonwood tree. Its seeds drift in the wind (definitely out of the range of the tree's roots/shade), and there's lots of seeds.

And yet the GP's point stands; the vast vast majority of them do not sprout, or, if they do, do not make it to maturity.