> Because extending the logic on such an idea, all people of North America, and most of the world, are immigrants.

But isn't that the point?

Does it actually matter where one person is born? Should it?

If your country has a system that "people from outside" want to exploit, then the error lies in your system, not whether the exploiter is internal or external.