Yeah, I've always quite liked that idea. Combined with marking things in different ways (funny vs informative, for example) I thought it worked pretty well.

There will always be issues with anything like voting, but it's an interesting approach.

HN also stops you from downvoting someone who replies to you, I think, which is another good idea.