Honest question: Do you think this wouldn't have happened if you where white?
I believe that when you are referring to rural, conservative areas anyone that is a stranger or strange looking (clothes, hair etc.) would have a similar experience.
Secondly, are you yourself white? I only ask because I am not, and I have had the experience of going into a store in a rural, conservative area, hearing conversations stop and heads turn as everyone stopped to look at the different-looking person walking in.