You don't have to, the votes will gradually adjust comment visibility based on its perceived (de)merit. Besides, an unpopular comment doesn't mean it's incorrect, e.g. the content might be useful but the tone irritating.

It was informative to myself at least.