If downvoting is done reddit style where it buries posts I suppose that would be good but also open the door for great abuse.

If it’s done YouTube style (thumbs down button) it just counts as engagement and promotes the post but without any visibility as to why.