It has been my experience as well that being a white guy means the other person - being a bully for the most part - feels that they can talk down on you endlessly with no consideration of your actual life experience.