I'm a man, nerdy, below average attractiveness, but a majority of my friends are women because I don't approach them trying to get something, and I've found that a majority of women are extremely receptive to being friends.
If you have a lot of women as friends, that's great. Your experience is not the experience of a lot of men. A lot of men can't relate to women on a platonic level, and that doesn't mean they're misogynists.
Young conflict-averse man starts life being a little socially inept. Tries breaking the ice with unreceptive women, typical human psychology sets in and this behaviour is generalised to all women. Now socially inept man continues through life with underdeveloped social skills while the people he wants to interact with are developing more complex social skills.
Who is to blame? Nobody in this scenario, Consider looking to why the man is conflict averse in the first place.