The issue with upvote rates is that they're likely to be relatively low, and arithmetic differences between low probabilities aren't useful, so we should really use the log upvote rate -- but people aren't good at dealing with negative numbers, so it should be mapped to color or size or opacity of some indicator symbol or something. Suppose you had just a little dot next to each comment, the opacity of which is 100 + k * <log upvote rate>. Suppose k=8; exp(-12) is like 6 upvotes per million views, so almost no comment's dot would totally vanish, but if you had a 5% upvote rate your dot would be at ~75% opacity.
I posted it as the discussion was about to fall off the first page, so here is it again for your reading pleasure :-)
---------------------
I have some suggestions that could help regulate the current issues. I'd change the way the karma is calculated in order to reward people who are consistently good rather than heavy posters. I suggest using the geometric mean of the post scores. As such, you wouldn't have to give a karma penalty to comment. A weak comment would reduce the karma automatically. I recommend the geometric mean since the scores are most likely Zipf-distributed. It could be calculated in the background as a running arithmetic mean of the logarithm of the scores, which shouldn't be too expensive. I'd display the karma not as an absolute number, but as color-coded rectangle. The color would be based on percentile of the score described above, or perhaps as a function of this percentile. The post count (or better, the post frequency) of the user could be represented as the length of the rectangle. You could then display this "badge" next to the user name in threads. Since it would be a continuous thing, it would not divide the comunity in have and have nots as the badge system introduced and retracted earlier. In order not to discourage people from posting, users below a certain threshold (say the 50th or 60th percentile) would all have the same value displayed. I'd also grant the various rights based on the basis of the percentile rather than based on hard numbers. Every time a new right is granted, users should get guidelines on how to use it, i.e. don't upvote based on agreement, don't downvote unless the post is offensive/shallow. Commenting in topics with little points could perhaps weight more than commenting in threads with a big score. This would encourage posting in less popular threads. Downvoting should cost karma. This could be implemented as a comment with a low score. You'd keep two counts: a post count and a downvote count. The karma would be calculated based on the sum of these numbers, but the post count would still be displayed. The value of a downvote would be calculated according to the number of post, otherwise it would cost less to people with lots of comments/votes. The karma of the voter may also be taken into account. The up and down votes should be weighted as a function of the karma of the voter. They could be implemented as a floating point number. Removing the number of votes per post prevents to detect a highly upvoted answer to a little upvoted comment. If the votes are weighted as described above, the vote count has little meaning anyway, but the post score could still be displayed in a meaningful way by using a color-coded system similar to the one described for the karma. I'd suggest two distinct scores log( comment score ) / log( highest score in the thread ) log( comment score ) / log( highest score ever at the time of thread submission ) Both of these could be calculated in javascript on the client. The real score could still be extracted with a browser extension, but I don't think it matters much. Making the first value relative to the max score at the time of voting would allow that value remains relevant as time passes, even if the community grows or shrinks. It would require to add a "max score" field to submissions, though.
Leave the point system intact behind the scenes, as is.
Collect the set of comments with a positive score.
For threads with < N positive comments comments, don't show anything
When there are >= N positive comments, show a percentile rank within that set. In the event of a tie, assign both comments the higher percentile (ie. "92% of the positive comments in this thread have equal or fewer points than this comment)
My initial guess at N would be 10. Anything less and the percentiles are somewhat meaningless and not really needed, as it's easy to skim.
The benefit of this approach: It's now a relative scale. Down voting no longer means "I don't like this" it means "the majority of other comments are better than this one". The net result will be less absolute value points for the best comments, as they will stall once they reach 90+% and they will receive far more down votes. However, there is now a community incentive to collectively sort the comment set, rather than award points.
You could have <(n positive comments/total comments) as a work around perhaps.
---------------------------------- The thing is that all mathematical rules end up being terrible at specific edge cases which are better served by another set of rules. In my opinion, the best possible use of mathematical rules are ones which are simplistic and easy to follow. Leave the rest up to the quality of the community.
(There is a different set of changes to cues and layout, which control social/behavioral action - such as removing the vote count, which I am not targeting.)
It would be nice if there was some way to re-nest comments, based on scores, while retaining their logical relationship (as replies). e.g. It could be done with lines or labels, but it sounds too messy.
Another way is too hide a comment if it has lower karma than a reply (using reddit-style [+] so it can be unhidden - or perhaps just omit it, and just use "parent" to see it).
The key benefit is that comments that only make sense as a reaction (e.g. criticism) to another comment won't be as interesting, and (hopefully) will get downvoted. This might lead people to change their voting behaviour, to also vote the parent up if they think it does contribute after all (it's a problem that interesting questions don't get voted up, only the interesting answers, because then it's lost down the page... and interesting questions should be encouraged/rewarded, IMHO). A downside might be that people will try to "win" the thread.
I really like this idea.
tl;dr: don't display parent if a child scores higher (still accessible by "parent" link), so they rank higher. Like slashdot.
I've had several comments both older and with less points than others at the same level, but that were still above the rest.
My understanding is that average karma of the poster influences it. I used to be more impressed but my average karma score has been as high as about 6 but most of the time sits somewhere around 2, give or take. One good debate where multiple comments of mine got substantially upvoted raised it that high and then it steadily slid (ie regularly changed if you checked it) until it was back down to around 2. Maybe other people have high averages that are more stable than mine but this experience has changed my view of average karma and made me somewhat less impressed. From what I gather, it's a moving, weighted average. It isn't the average across all comments for all of your history. I can understand the logic of that -- if average for all comments were used and you have been here forever and your comment quality has gone up, a million old comments that were never voted up could prevent you from ever seeing any substantial improvements in that area. But, like all such things, it's a bit of a game in some sense.
BTW: edw regularly has amazing averages - all the avgs are much higher than usual now, I suspect because of karma inflation due to not seeing scores (this happened last time pg experimented with it).
And yes, it influences the ordering of ones comments. When I said "approximates", I really meant it. ;-)
http://blog.reddit.com/2009/10/reddits-new-comment-sorting-s...
Not sure how effective it was.
a star, orange dot, bold title, anything.
that said I like HN without points.
As it is, I find myself commenting less to replies on my own comments, because I'm not invested in furthering my argument - I can't see how many votes those replies got, so I don't know if I need to keep arguing my case.
(That could be a good thing or a bad thing; a good thing if I'm wrong, a bad thing if I just didn't express myself well to start with.)
While a lot of insight can indeed fit into a sentence or two, it still figures to me that those who take the time to write long comments are probably contributing more to the thread than those who leave short replies.
Anyhow, it's just a heuristic that springs to mind.
I also dislike the lack of points because I find myself voting things down more than I would before. In the past, if something were at 0 or -1 points, usually I'd think, "They probably get the point." Now I find myself downvoting things that probably don't need one more down vote.
I agree with this, may be displaying a color coded dot in place of the comment score. The color can be computed from a log function of the score, and we only need a few colors to indicate the different types of comments without giving too much information away.
Ah, came here to say exactly this! The difference between 100 points and 120 points just isn't as interesting as the difference between 1 and 10.
Displaying something like floor(ln(points)) instead of points would be the way to go.
I hadn't thought about this, but you're absolutely right. I think having the numbers on other people's posts does facilitate gaming the system quite a bit more. I'd be curious to see what the actual distribution is on points/comment, which might inform efforts to implement a logarithmic scaling on the effectiveness of upvotes.
No matter what happens with having other people's scores, please keep the option to have our own scores displayed. Having feedback on what the community finds upvote-worthy is definitely helpful.
If there are so many skimmable comments in a thread that not having points gets to be annoying because you have to sift...well, then the experiment has failed.
But if on the other hand there are just plain fewer comments, which does seem like a possible outcome in the mid-range, and those comments are of higher quality...well, experiment has succeeded!
Points trigger a certain response among some people. The ones that get sucked into WoW for example simply because they have this impulse of getting all the best gear. Works with points as well.
In that case, you might as well just have numbers, since you can correlate numbers with colors, at that point.
I would rather see colors indicating the "heat" of a comment. That is, its numbers, in relation to other numbers in the thread. In other words, you wouldn't have the following:
Blue == 5+ Yellow == 10+ Orange == 20+ Red == 50+
Instead, you would have:
Blue == average score in this thread/topic Yellow == 5% higher score than other posts in this thread/topic Orange == 10% higher score than other posts in this thread/topic Red == 20% higher score than other posts in this thread/topic
This would help provide a more meaningful context for what posts are rising to the top in that particular conversation and the highest would always rise to the top. The top 20% of comments would always be the top 20% of comments, even if the lowest scored comment in the entire discussion is +10 points.
Anyway... really like your idea :).
Was going to write something about colouring entire threads or sub-threads as interesting based on the total sum of points or something else compared to everyone else. Would certainly help finding interesting stuff to read in 200+ comment threads. But not too sure how it would work without being invasive.
The idea would be to highlight any path through the comment tree where comments are strictly increasingly valuable. So, for example if someone asks a question and gets a really good answer, the scores on both comments will be shown. The score could even be shown on an even better reply to the answer. But if someone trolls the really good answer, neither the score of the troll or that of responses to it would be shown.
A further improvement on this might be to use a time weighted value (where newer comments have more weight) similar to that already used in comment sorting so that comments have a chance to gain value instead of being pitted directly against an already popular comment.
Also, I've been planning to write a longer blog post on the following, but given that I've had no time lately and am not likely to have any soon, I'll just float the idea here.
One idea I've had that I think might be interesting is dealing with upvotes or points in terms of logarithmic scales. That is, it takes one upvote/point to get a comment from 1-10, 2 upvotes/point to get from 11-20, etc. (Exact numbers would have to be scaled, of course.) I find that going into a thread an hour or two old and seeing comments with 50-100 points is a major disincentive to commenting, even if I have something to say. That comment or couple of comments and their resulting threads are going to make sure very few people ever read what I've written. An appropriately scaled log-scale system might make it so that really really good comments still get really really high scores, but so that others (which might have simply come too late in the discussion to be competitive on a raw-point scale) still get a chance at being seen.
(One related idea would be to make the point-approximating graphic log-scale even though the points themselves remain the same underneath.)