Those are all three bad examples though. Call any black person "nigger", any gay man "faggot", or speak to a special ed teacher about his "retarded" students and on the whole all will be offended.

The fact that retarded was a clinical term is irrelevant, similarly I think "nigger" derived from some matter-of-fact observation. This is really just another manifestation of your idea that root causes matter. They don't -- not as much as impacts do, both legally and for many people morally as well. Also, again, reciprocity is not a requisite here -- there is no fair use policy for offensive words or behavior.

Whenever someone says "if they can use it so can I", I have to wonder "why would you want to?" Why would you ever want to refer to gay people as "fags" or black people as "niggers", even if they do call each other in such a way.