I think the threshold for flagging is lower than for downvoting, but many people don't realize you can flag a comment because you have to click on the timestamp to see the "flag" option.

In some cases, flagging is appropriate.