It should be also mentioned that on most days it only takes a handful of votes to get to the front page.

It might take a lot more with downvoting!

It would certainly take more to get to the front page with downvoting. I think the trouble with downvoting is if you have a polarizing story that is interesting the vocal minority could in practice downvote stories that don't align with their worldview. Thus censoring an otherwise merit based story economy.

I was toying with different downvoting schemes but all of them seemed flawed in a fundamental way:

You could have downvotes after a certain threshold.

You could have downvoting carry a different weight based on how much karma you have.

You could have downvotes only after a certain time has passed thus making stories with polarizing opinions not get squashed immediately but still allow for the opposition to have their voice heard.

You could have downvotes only enabled on submissions from members under a certain karma threshold.

I am sure there are many more ways to do the downvoting but I think there is a more fundamental problem with downvotes in that it creates a kind of hostile community. It might be worth having the occasional dumb story come to the front page to keep this place a civil place. Downvotes in the comment section should be reserved for a person that does not further the discussion or is overly rude to another member here.

What about N downvotes per day, possibly based on the log of your karma? This reserves them for something that really bothers me, but limits the power of a like-minded community. It might also help to display the downvotes in red next to the total, since a downvote would be more scarce than an upvote, and may be more informative.

Not a bad idea... I like the attenuating nature of the log of the karma. It would probably need to be coupled with some of the other ideas in the parent post to be fair to new stories with only one karma unfairly down voted by one or two dissenting voices.

In that hypothetical situation it might be interesting to see both upvote and downvote numbers to gauge a 'controversy' vs. complete lack of interest (on both submissions and comments). Or perhaps listing 'score' and 'votes'.

Yes, but I actually sort of prefer the way this incentivizes sharing of interesting links with the community. That is, the only way to get the stuff you don't like off of the front page is to share something better!

Moreover, it indicates that getting the stuff you don't like off the front page is not something to strive for. If you don't like it, ignore it. It is a sign that at the moment, the community here has found nothing more interesting to show you.

I can't say I've seen a link I disliked so much that I felt the need to hunt down something to help drive it off the front page. This incentive seems dubious, and if real, the benefits dubious.