I still think the perceived reward of having high ranking comments has to be removed somehow.
What do you think bruce?
Karma should be displayed to the logged in user in a vague fashion: "Karma: good." The numeric score can be maintained on the server, but never displayed to the user. Downvotes and other abilities would simply show up one day when you've passed a certain threshold, but the user should have no idea how far along they are, as we're trying to dissuade the people that want those sorts of powers (and would abuse them) from getting them. Meanwhile, negative karma could start rate-limiting you past a certain point, such as only allowing 3 posts/day.
I'm still of the belief that the only way to ensure high quality discussion is to discourage commenting for it's own sake. Everybody does not need to contribute to every article, and having a numeric karma score only legitimizes posting for its own sake.
To reiterate my support for a previous point: the only way to fix the karma system is to kill the ego's involvement in it.
Simple systems, like the current karma system, can certainly be gamed, but with each iteration general value is lost. Displaying comment scores (which I prefer) can be abused. So that's gone, and a data point with it.
Keep them, don't keep them, I don't think I'll lose much sleep either way. But as a general rule, removing data or functionality from the many, to try and prevent the excesses of the few always invokes the ire of unit ended consequences. It's almost never an improvement.
IMO Removing karma won't improve the quality of the conversation.
Equally, anonymous, or pseudo-anonymous, comments give the commenter more freedom to make unhelpful comments.
Comments are tied to the identity, which goes at least some of the way towards them being self-moderated.