But you seem to miss the point that some people are dicks, but are mostly dicks to women, or black people, or disabled people. They may not realise they're being dicks, but other people will recognise elements of dickishness or outright dickery.
And also that fighting the dicks makes life better for everyone.
I find it weird that when women say (for example) "We are sexually assaulted so often a separate women only London Underground carriage would be welcome, until law enforcement and society can pressure offenders to stop offending" they are met with hostility and denial, rather than a willingness to try to understand the severity of the problem.
Concerning your last point - I'd guess they're met with hostility and denial because they legitimately don't believe the issue is as widespread as many are led to believe.
As for why we single out people who hate others because of skin colour, but not because of say hobbies: A large number of people have been murdered (or beaten, or denied employment, or denied housing, or etc etc) because they are black. I don't think I can find more than a handful of philatelists who were murdered because they collect stamps.
I think the touchiness around discrimination based on skin colour is understandable and based on plenty of evidence.
Yes of course, a lot of people aren’t killed for collecting stamps. But I don’t think many people today are killed specifically for being black or gay either. It does happen, yes. But I believe the overwhelming majority of people just carry on with life. We should all remember that whites vs. colored fountains were deemed lawful by the government.
I truly believe most people today just want to go to work, do their job, and get paid regardless of the gender or color of the people around them. And honestly, what more would you expect? We live in the country with the greatest amount of diversity you can’t escape. Americans think Americans are the least tolerant and my travels to other countries and interactions with different people shows me the exact opposite.
Edit: I say this as someone who grew up in the South. I lived around racists but most of that stays in their head.
This is all well, until these same women start vehemently opposing male-only spaces (e.g. men's clubs, male barbers, ...). Sure, the reasons might be different, but just as women are concerned about being touched by men (physical), so are men concerned about being judged by women (psychological). I think both deserve to be taken into account and made a reality.