I much prefer being in more diverse areas, but I don't think where I grew up was somehow bad or wrong. It is a very middle class place, so it's not like the minorities were being pushed out, there just weren't many non-white people that wanted/want to live there! ...but to be fair, a lot of white people don't want to live there either, it's quite boring.
Further, if housing discrimination was still going on in the 90's and 00's, I have no idea where these people were being "pushed out" to. The closest towns/cities with significant racial minority populations were at least 250 miles away...the city I grew up outside of was 90% white (currently 82% white) and there were/are plenty of undesirable locations there. It's middle America, there are just a ton of white people (sorry?).