I think the existence of the downvote is good so the community has a tool to self-regulate and avoid the guideline-breaking-but-not-inappropriate comments, like the low signal-to-noise ones.

I am against the use many people give it to silence people they disagree with, but the problem is, what's a good alternative UX?

I think keeping the functionality while presenting it as something different than the opposite of the upvote might do (maybe use an alternative counter, too?), but on the other hand it seems to just overcomplicate things.

The functionality as-is might not be perfect, but it might be better than any alternative I can come up with.

Does sorting by upvotes solve it without needing downvotes?

Sorting would be at sibling level.

But what do propose to do with the graying out feature? Discard it? or make it like the flagging feature, which doesn't have an opposite action?

I think it's useful for it to have an opposite action, but then that would mean we have votes for sorting and separate votes for graying out. That seems needlessly complicated, which is what I meant.

My initial inclination is to abandon the graying out.

Upvotes may cause the post / comment to float up in relation to it's siblings, but that's about it, other than flagging.

I also think it would be interesting to have a separate agree/disagree signal that's independent of quality rating.

I can disagree strongly with a well thought out and quality comment, and vice versa. It would be nice to have independent signals, so I could conveniently communicate "I disagree with you, but value the discussion"