And why should fear be the only cause to bar entry? Is that the standard you'd use for your home? I'd start with close friends and family, myself, then go from there.
One doesn't (and shouldn't) run a country like a house.
You get to limit your house to friends and family. Have at it.
You don't get to limit your country to that. Hell, you don't even get to limit your property to that, if you've got a sidewalk on it.
And much how the members of a household influence who to let in, the citizens of a country decide that as well, via voting.
Or do you consider yourself a higher authority on what they 'get' to decide on?
Same with immigration - courts have blocked plenty of enforcement measures and violations of human rights like long-term detention of children. There's a reason "majority rules" isn't the only way we do things in the US and most other democracies.