You make a good point. The "I find X difficult" is a 'disability' trait, and disabilities are only really relevant where they conflict with the expectations of wider society. In all cases it's really the social exclusion that's the problem. If someone has a trait that either can't be changed, or takes exceptional effort to mask, it shouldn't be on them to do the impossible. If however it's just habit, or actual lack of effort to change, that's a completely different thing.