I agree: I don't think that the theoretical oligarchy should have complete control. But moderation control is different. We might get a more civilized community if the upvotes and downvotes were controlled somehow.

It won't happen, but I think it would be an interesting experiment.