"You cannot do / say X because you're skin color / gender / sexual orientation Y" is racism / sexism, end of. "Affirmative Action" (which is doublespeak for "we prefer minorities") is racist/sexist too (and often referred to as being reverse racism).