Abusive downvoting bothers me on occasion. For that, there's Reddit.

But ambiguous UI elements aren't going to have that effect. People will generally either misuse them habitually or ignore them. The human mind is awesome at filtering out noise.