This is true, but what has the higher net social utility: a system that treats the 1% of social deviants equally at the expense of a loss of social cohesion, or a system that treats the 1% of social deviants poorly while maintaining a friendly social environment for the 99%? The answer isn't immediately obvious, and depends on what degree of deviance the local culture will tolerate, so I could see it going either way.

However, I do think the concern is a bit overstated; especially in northern towns, the primary cultural identity shared by the inhabitants would most accurately be described as "outdoorsman" rather than any religious or ethnic affiliation. There's more to it than that, but overall it was pretty accessible as long as you were willing to adapt.