There's a heck of a lot of room for variation between "minorities and poor" and driving up the crime rate.

The blue collar dude with five cars and a project boat in his yard is totally unwelcome as far as the "muh property values" types care. He's not driving up the crime rate or leaving used needles around though.