also known as https://en.wikipedia.org/wiki/Lump_of_labour_fallacy

It's an oversimplification to portray the "other argument" as asserting that everything in society is strictly a zero-sum game. In some important ways, it is; and in some important ways, it isn't. People may be wrong about those details, but that's not the same thing as the common quip "it's not a zero-sum game", which essentially portrays the other side of the argument as dunderheads (a trap that well-meaning people who believe they are on the "right" or "moral" side of an argument very easily fall into).

I.e. no, it's not a zero sum game, but it is a complex system with inputs, outputs, some finite resources, and lots of human subjectivity and desires. Concrete harm easily comes from bad immigration policy (regardless of the "direction" of the policy's badness).