That would lead to general inflation. You need to give comments on both directions.
What I'd like to see is article downvoting. As it stands there is no user-inflicted punishment for posting a sensationalized or otherwise terrible article
Which just shows how few people up-vote articles. Getting front page of HN easily gets you 5-10k pageviews if not more, yet your average top story has 100 votes. Most people consume without contributing in any way.
I'd just like to see how that played out.