The fear is mixing the x and y axis of opinions. Most people can probably see why something like party, ideology or nation can be damaging. Not talking about those things generally doesn't prevent people from having a certain opinion on another level. But then you mix that with things like race, gender and class where wanting to talk about it generally (but not always) constitute an opinion.