yea, that's a good idea. I would back this idea.

It would prevent people from hunting down someone, and downvoting all of the posts without penalty just because they didn't like them.

Though I'm not sure how much of a problem that is here.