Minority should be respected just as much as the majority. Downvotes make "less popular views" also less readable and promote a "tyranny of the majority" which is bad.

If downvotes did not affect the visibility and were called "disagree" it would be fair. Instead they serve as a mob's club in my opinion.