The reality is predatory men already invade women's restrooms for perverted reasons. You don't think they would not be emboldened by being able to shame people for being transphobic to increase their chances of getting away with whatever they are doing?