Because it is discriminatory to use comfort as a criteria to exclude people in the social sphere. It's like trying to ban your ex-partner from a local bar because bumping into them evokes strong feelings like anger or jealousy, and therefore makes you feel uncomfortable. This is not only wrong but it is an immature way of dealing with personal issues in real life.