But in an environment where that doesn't matter anymore, it's only natural that sperm "health" will consequently deterioriate. Natural selection doesn't select for "the best", it only selects for "good enough". And if "unhealthy" sperm that can't move sufficently is "good enough", well so be it. Especially if we take factors like in-vitro and artificial insemination into account, those really remove the "health" of sperm from the equation of pregnancy.
I agree that in-vitro and artificial insemination could remove an important feedback mechanism for sperm evolution, and that would be a cause for worry.
Which by itself is fine, really. Sperm that fails to reach an egg means the genes that made that sperm do not "deserve" to reproduce. However, if people want to make a fuss over low birth rates, and this is one factor behind it, it's going to get attention.