> I don't think many blacks care of being a "small minority" on a posh white neighborhood...

We do. Besides what a sibling has already mentioned regarding some small cultural difficulties, there's the whole "police killing us all the time" thing which is much less likely to happen if you're in an area where being black doesn't make you stand out