"Holding people responsible" means punishing, in one way or another. Not doing business with them, trying to get them tossed out of a group, get them fired, shamed, etc. And who is going to do this to whom? Those with the popular opinion would be doing it to those with the unpopular opinion.