I've caught myself before sometimes being more careless with the downvote button if the comment was already grayed out. After I realized this risk, I now attempt to be more mindful.

So it is a thing that happens, at least to a certain extent.