I experience a similar thing. I'm on the spectrum, and with new friends it's always an anxiety inducing countdown until they identify me as an "alien", after which we are really no longer friends - I am an other, to be ridiculed and/or avoided. It's gotten to the point where I don't really try to make friends unless I can detect that the other person is also on the spectrum.