It isn't about people being ageist. The fact of the matter is that if you look like a bit of a weirdo and you are and older guy people will look down on you and find you suspicious. It doesn't matter if you or I think it is fair or not.