>Downvotes result in a ban

They do?

>Also votes were supposed to control quality, not agreeability.

That's a completely orthogonal problem. Regardless of whether downvotes are being "correctly", the "cancel" aspect is still there.