That would be more like downvoting an account or a person. Downvoting a comment simply signals that there's something wrong with the comment. If the next comment is fine, it's welcome at the lunch table.

It's true that comments sometimes get downvoted unfairly, but I'm not seeing that in this thread. In such cases, though, the thing to do is give them a corrective upvote: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor....