I work in a Psychology department that has some members who specialize in the study of various kinds of bias, and I can assure you that unconscious bias is very real, and measurable in ways that I honestly can't find any reasonable way to claim measure anything else.

Look up the Implicit Association Test for a good example of this. I've taken it, and while it wasn't exactly a shocker to me to discover that I have an unconscious bias against black people (I grew up in an area that had maybe 5 of them, and despite my very liberal upbringing, our society and culture still label black people as "other" and "dangerous"), it was eye-opening to be able to see the effect so clearly.

IATs are a good example for the immaturity in this area. It's a test designed to create the desired results. Switch the good/bad/black/white categories with any other two categories and I will take the bet you will get the same results.

The Implicit Association Test is - shall we say, non-scientific. Because the results depend on the ordering of the associated categories[1]

Psychology has too long been a field where people draw a conclusion first and then create some experiment or hypothesis to support it. It is only now with the advent of analytics that we are shedding light on this quackery.

[1] http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=22...

I posit that your bias is more likely due to you not being black rather than not being around them while growing up, and your reaction to a person of color is an a priori programming which simply tells you "not my tribe." Feelings following that, however, are what you as an individual and, to a lesser extent, what society dictate.

I had the same experience after moving to the US for the first time. I'd never had a personal interaction with somebody of color, and all my views had been shaped by what I'd seen on American TV (typically not flattering, in the 80s/90s).