I suspect it stems from a feeling of fairness: I have to follow the rules, so how come you get away with doing whatever you want? That's intuitively not fair.

Now I don't agree with that from a consequentialist point of view, but I could understand the feeling.