Your position is called a 'lump of labor fallacy' by economists. A job filled by an immigrant still creates demand in the local economy (for housing, food, etc.) which in turn leads to more job creation. It's not a one-dimensional tug of war between jobs and unemployment statistics.