I don't disagree with any of that; every gender/race/etc has SOME kind of privilege attached to it.
The point that you're [ missing | skirting ] though is many folks use the existence of privilege to ignore/bully/hate (eg, my YT link above, the white professor who got berated on campus, etc etc).
Yes. Things can flow the other way as well.
Did black kids pick some fights with me in school because I was white? Yes. It's a real thing that happens.
It's a fallacy, of course, to dismiss a belief just because some of its believers are assholes, even if they're being assholes in the name of that cause. I mean really, no group or belief on this Earth could ever pass that criteria.
If it helps to hear this from a white male who is a believer in recognizing white privilege and male privilege: being white and/or being male are sometimes disadvantages too!
In most of America, you can't be male and teach young kids these days. People (parents) will wonder if you're a child molester, and get weirded out by the prospect of you helping kids go to the bathroom even though they'd be fine with a female teacher doing the same thing.
Recognizing something like that isn't incompatible with recognizing male privilege.