It works in the opposite direction too, though. If you live in a place where a given sort of people is scarce, it doesn't matter how hard you try to be unbiased, you internalize prejudices based on your lack of (or minimal) experience with them. Then someone does come along and they don't fit in, and they are set up for failure, and try as you might, you (subconsciously) associate whatever group they belong to.