I believe that rebellious '-1 downvote' comments will be far fewer than the actual downvote so original idea seems better. In fact such rebellious comments should be deleted/flagged in case they don't provide any additional reasoning behind downvotes similar to what happens on HN.