Wouldn't that be an argument that an alien race wouldn't annihilate us, rather than they would? I mean, what good is a dead planet for genetic diversity (and, I suspect that if enslavement were the goal, humans would resort to extreme lengths to try to prevent it, up to and including destruction of large swaths of our own planet)?

But, I don't think enslavement or a human zoo is a very likely motivation for a species advanced enough to travel light years to get here.