It's more likely that folks who claim to not have these associations simply haven't identified their unconscious biases yet.

There is always a reason associations between mental traits and physical features exist, even if you're not consciously aware of it.

EDIT: And to go a different direction, if someone truely doesn't make such associations... Now that they know some people do recognize those associations, just be polite and don't make such comments about people's appearance - be it skin color, hair color, height, weight, etc.