Downvotes don't hurt me. They stop being a disincentive when there is no clear reason for them. It's often people just misunderstanding, misinterpreting or misreading comments and the replies keep flowing anyway.

It's not like you get paid for getting upvoted and a making any kind of joke is usually the fastest way to a downvote.