I think education, wealth, urbanization, the pill, the boob tube, condoms, all the reasons for below replacement fertility can be seen as a virus which attack the reproductive system.

A few people are partially or fully immune. (You don't have to be Mormon to be well off, educated and absolutely need to have 4+ kids) and over time their numbers will come to dominate.

I hope we've colonized space by then.