> there should be a rule on HN against downvotes without responses

Downvotes are specifically for things that do not support productive discussion, so downvote + response makes no sense.

EDIT: If HN had a private message facility, I can somewhat see a case for private feedback with downvotes, though I suspect that would also end up unworkable.