I think this is an example of a mutually beneficial filter. To me someone setting up separate office hours for white men and for everybody else sends a strong signal: "this is a person you don't want to deal with; avoid!" On the other hand, it makes them more likely to meet people who endorse the same kind of thinking, white male or not.