I think moderation should be done in the open. It's more an issue of people allowing their content to be filtered by others, I think. A downvote is only as powerful as the user's moderation privilege.