I think it would be possible to limit the use of abusive down-voting as a censorship mechanism. You'd need a list of what kinds of posts can be down-voted, and enforcement where people who moderate badly lose the ability to moderate.