Can you guarantee that a large, rapid influx of immigrants won't lead to a poor underclass? Because that's as much of an issue as wherever they came from...
In no way is that a "large, rapid" influx. This whole issue is just political football and scare-mongering.
You're right, the US has a low rate of immigration and should be able to handle it, but that's not necessarily the case.
Also, 0.3% per year amounts to nearly 8% over one generation, not including differences in fertility between immigrant groups and born Americans.
And you're right about the 8%, but since only about 10% of immigrants to the US are Muslim, really we're talking about 0.8%, over something like 25 to 30 years. That is so completely disproportionate to the hysteria over immigration (literally the number one issue for Trump voters, IIRC) I can't help but conclude that it's a purposeful scare campaign.
Also, it's not often that any immigrant group is the boogeyman for more than a generation. It used to be the Irish, then the Polish, then the Puerto Ricans, and so on. A huge fuss is raised, and then they assimilate, at least in the US.
I can't find a non-paywalled link ATM (the ridiculous politicization of this issue has made it nearly impossible to get signal out of the noise), but I've seen studies that show that if you control for poverty, the link essentially disappears.