The vast majority of the people accused of being racists aren't actually racists. But even if they were, it's not your place to decide who gets to speak and who doesn't.