I think downvoting should cost something. Maybe not a whole point, but something.

If shutting someone else up (and let's not mince words here... that is exactly what you are doing) isn't worth even a token cost to you, maybe you should reconsider whether shutting them up is truly justified in the first place.