The problem it that they're "punching up" against people that could very well have had their own challenges in life, some of which may be even grater than group doing the punching. Don't expect a white guy that grew up in a trailer park with alcoholic parents to feel like an upper class black woman is punching up to his privileged position in society.