Because the dread god Ki'urnac wills it so. Wait, you don't know about Ki'urnac? Oh, well then it's good I mentioned him, because now you'll now to follow his will so he doesn't torture you in the Dark Eternities.
More seriously: Most men don't feel particularly privileged. A lot of them feel downright oppressed (not because they're men, per se, but just because life is hard). If you suddenly tell them, "You're wrong, but you don't know it because you're so privileged," it's like bringing up Ki'urnac. They have roughly the same level of belief in the statements "Ki'urnac exists" and "I am privileged." Unless you're planning a lengthy proof of Ki'urnac, they're probably going to write you off and feel mildly offended at your pushiness. That's why I think it's better to point out the actual ways in which people are privileged, rather than just tell them they're privileged and thus they should distrust their own minds.
The point isn't if it's intended to be sexist, or if it is that big of a deal in and of itself. It's a tiny pebble, one that most of us wouldn't notice in and of itself. It's as if most of us have shoes and we're walking over some gravel. Someone who was barefoot would notice every pebble a lot more.
I think that's why mentioning male privilege is important. If we don't think about it, we're more likely to throw a pebble on the ground without thinking of the consequences. More importantly, if we are unaware, we're more likely to dismiss someone who has a valid complaint about a small issue, because they've been dealing with a thousand small issues that combine into one large issue.