FWIW, I agree and I hear both what you're saying and what @crazy_horse said. My perspective is coming from having lived through that perceived difficulty and hesitancy and come out the other side. @crazy_horse isn't there yet. I was terrified of downvotes early on, and I admit sometimes they still bother me. One of the reasons HN gates the downvotes, I believe, is so that you have time to go through the process of introspecting and reflecting on how you're handling yourself before you start reacting negatively to others.

For me, it actually changed my mindset before I had access to the downvote button. The result is that I think more carefully about how I phrase things, and also that I've never used the downvote (at least not intentionally, but it's easy to fat-finger these teensy buttons on mobile devices) because I remember how it made me feel, and because upvotes are usually enough to percolate the good stuff upward. Having said that, I've seen a lot of cases over the years where the downvote button did good things to a thread, it often does a good job of sorting by relevance and helpfulness, it often lets people know when they're getting out of line, and for me personally it helped me be a better commenter. But I first had to get to the point where I admitted I had room to improve, and that took time and some downvotes.

Lately I've been thinking about changing my personal policy and using the downvote button every once in a while, because over time I can see some of the upside to the downvote. So far though, I've often been glad that other people do it so I don't ever have to.