As for more detailed buttons - I think that would make it too complex. Slashdot has established the same concept, probably based on the same reasoning (mostly to distinguish between "just" funny and actually insightful comments). Can't speak that much about how successful it is, maybe somebody else has more insight here.
I am working on a site now that is heavily based on passive activity voting, users have no idea how their actions are being used by the site to rank itself. Their inputs have much less weight than purposeful actions where both parties are aware of the event.
It is almost like you need an agent that votes in certain public ways (sentiment analysis,summarization,etc) to decide if comments snarky, funny, off topic, etc.
Passive voting sounds interesting, though - would that work via Javascript? How would you determine what the user is focused on?
Finally - I think it's important not to get ahead of ourselves - because I think that's what happened at Slashdot. Talking about new classifications is all nice and well, but it's useless when you don't have a clear plan what it should result in and how it would engage people. You need a clear use case.
For instance - my proposal about an off-topic discussion button is pretty straight forward. People would see the immediate benefit: Hey, this can help us make conversation less wasteful and save me scanning time as it separates the cream from the crop. I think it wouldn't have a problem enticing engagement, too - nothing drives engagement in nerds like being annoyed, so they'd click that button.
On Slashdot, I can theoretically reduce a conversation to exclude all the funny-only comments so I'm left with just the cream. But seriously - how often does anybody use that? Once you give people elaborate personal filters, you actually end up having them worry whether they're missing something. Which is why they can only be consistent and transparent. If I reduce a comment thread on Slashdot to exclude the funny business and see that one of the nodes has a HUGE conversation, I wonder whether I should check it out and maybe end up annoyed because it's just a chain of memes after all. If there was a way in HN to mark things as off-topic and have that a clear process that is always applied, I think I would be more confident in skipping. Mainly because if it was broken, the community would complain until it is fixed.
I don't have a solution. I am not sure there is one or that it could be "fixed"
Maybe semi-intelligent Eliza like agents that constantly clean the place up.
Stackoverflow has done a pretty decent job I think. Wikipedia could use some work.
----
I think the base set of rules need to be codified by the system. The users can guide each other and the system, but the system itself should be stable.
But if you had a button for each piece of information it would look like
For bonus points they would be in a random order in the UI so people wouldn't develop muscle memory voting on comments.