Who says the two are exclusive? You can go to any US planning meeting and find at least one person who alludes to the wrong type of person moving in resulting in loss of "neighborhood character".

Racist and classist concerns also influence property value. Blockbusting was a real estate practice that worked https://en.wikipedia.org/wiki/Blockbusting