It's important to keep in mind that--in the absence of downvotes--scores only mean that content is relatively more relevant than other content surrounding it, not that it is absolutely relevant to our audience. The current voting system encourages diversity, and it's diversity we get.