sometimes people present reductive versions of an argument and drop relevant contextual information and then complain that others respond unfairly. that said, favoring deportation for unlawful entry - wholly absent of any context or further data - doesn't contain much information to assert 'racist'.
however, there are serious 'racial implications' of various immigration policies, such as the 'papers please' arizona law, which says officers can harass people simply by being 'suspicious that the individual is an illegal immigrant'. if a person's status is unknown, the means by which such suspicion is derived would seem to principally lie within racial profiling and similar tactics. among other problems, this profiling creates an unreasonable burden on american citizens that are hispanic (particularly those of mestizo heritage).
Probably because of the massive amount of racist rhetoric used in support of that position. It's not an inherently racist position, but when such a large portion of arguments in favor of the position use racist rhetoric it's hard to avoid being lumped in with those arguments.
So when you come out against illegal immigration, your stance can run the gamut of not racist at all to incredibly racist. It just turns out that most of the time the statistics simply don't line up for illegal immigrants committing more crime, stealing jobs and so forth. Which then makes the argument not about fact, but about feeling. Which then tends to be more inclined towards racism.
Of course race has nothing to do with it, I couldn't care less what color you are , if you're here illegally you need to be deported. And it's not just me, but I constantly see others who have the same beliefs getting accused of being racist.
And that's just one example. I've seen people be accused of being sexist just because they didn't like Hilary Clinton. It was actually really common for liberals to accuse opponents of Clinton of being sexist regardless of the total absence of any evidence of actual sexism.