Yeah, downvotes are weird in that way. Comment sections that marks posts and comments with more downvotes than upvotes as "low quality" also seem to encourage "pile-on" behavior.