There are two cases. The first case is if your comment deserved a downvote. In that case, yes, one downvote on one comment makes it difficult to adapt. But if the behavior that caused the downvote appears in other posts, those will get downvoted, and you should be able to discern a pattern in what is not appreciated.
The second case is if your post did not deserve a downvote. This happens, particularly as our community grows. Some people downvote comments merely because they disagree with what the person said. We don't want it to happen, we (the community) try to correct for it when it does, but it does happen. And in large threads, such posts can get "stuck" at the bottom in a place where no one sees to correct it. In that case, don't worry about it. It happens to everyone.
How do you know which case it was? Keep posting and you'll be able to find out.
Thank you, this makes sense, and is appreciated.
If someone wants to explain a down-vote, they'll say it in a reply to that comment.
Requiring explanations will merely result in meaningless deluges of "not helpful", "bad", "this is a comment of a type I don't wish to see on HN", or people C&Ping long political rants.
Getting no feedback other than a downvote doesn't help me adapt to the community. I may not agree with why my post wasn't accepted by the community that is HN, but at least I'll have more information to work with.