See, though, this is the thing. It's great that you're not being all "lol male tears", have a cookie, but you're putting a lot more effort into defending your conception of feminism, and into drawing a distinction between yourself and "lol male tears" and making sure we all know you're not like that, than into addressing the topic at hand. That's what is getting you downvoted, not some imaginary knee-jerk reaction to a word.

It's not accepted to derail a women-centered discussion with "but patriarchy hurts men too!"; why do you expect anything less in derailing a men-centered discussion with "but feminism is for men too!"? Granted you're not being That Guy in the usual direction. But you're no less being That Guy for it. Maybe consider not being That Guy.