And through natural selection, those kind of people (who care, that is) will dominate the human genetic pool, while the ones susceptible to falling into the provider role will systematically be weeded out, like it or not!
For another thing, people who care might be less reproductively successful on average. This is a bit more fanciful, but IMO is more plausible than it sounds. Perhaps people who care about that stuff won't have as many children because they're more picky, for example, or for any other reason that might correlate with that preference.
And that will only provide a more fertile ground for proliferation of a specific kind of genes, don't you think? (Of course, for that you must have to believe at least a little bit that genetic behavior is not entirely replaceable by parental education.) Currently, if having a harsh life as an abandoned child is something of a lesson to prevent a similar experience in one's offsprings (which are cared for), then through each adoption we're just preventing/affecting a natural homeostasis.
I agree with you in that I don't care, myself; but I don't think it's fair to judge the people who do care.