I posit that your bias is more likely due to you not being black rather than not being around them while growing up, and your reaction to a person of color is an a priori programming which simply tells you "not my tribe." Feelings following that, however, are what you as an individual and, to a lesser extent, what society dictate.