News for hackers. Stuff that matters.
Re:How Not to Sort by Average Rating (2009) by danialtz on Thursday July 09, 2015 @10:20AM (#9857956)
I have fundamental problem with democratic voting systems. Whatever the general view likes will tend to come on top, hence cat pictures on reddit. The most philosophiycally elegant solution I've encountered so far is the "quadratic voting" (see https://news.ycombinator.com/item?id=9477747), where every user has a limited number of credits to spend per time-period. Every vote will have a quadratic cost.

Assume user has/obtains 1000 karma points a month. If the user merely likes a post or not he gives his 1 vote, which cost him 1 karma. If he strongly wants one post up he can spend a maximum of 31 vote on it. This way the minorities will have also extra influence on the voting process.

The requirements is that each user have 1 account, e.g. maybe by some form of payment for 1000 karma to avoid fake voting fraud. Maybe using bitcoins if it picks up to avoid privacy problems.

Do you think this will hinder the workings of sites like reddit?

Re:How Not to Sort by Average Rating (2009) by bwy on Thursday July 09, 2015 @11:07AM (#9858240)
What I always thought was that there really should be some user-based weighting system. Like, if a user upvotes 90% of the things he sees, his upvotes are probably worth less than upvotes by someone who upvotes only 1% of the posts he sees.

Same thing applies to things like Yelp reviews. Maybe a user with close to a 5-star lifetime rating average should have his reviews "renormalized" to 3's because his standards are probably just lower than the guy with a 1-star average.

The problem is that there are so many other factors here (maybe the 5-star average person only visits (? or just reviews) really good places). Maybe the crazy upvoter just spends more time reading each page on Reddit. These are complicating factors that are hard to predict and if the simple case is working, why try? If there were a simple, clearly better way of voting/rating, it would be done.

Re:How Not to Sort by Average Rating (2009) by marcosdumay on Thursday July 09, 2015 @01:07PM (#9859211)
Yes, Bayesian weighting. I feel that every kind of pool needs Bayesian correcting for calculating how much information does an opinion really carry.

The problem of people only visiting isn't really a problem. If a person only visits good places, he'll be perfectly able to differentiate those good places from each other, and can say one of them is bad. Yet, somebody that visits good and bad places has a saying on what places are good, and what ones are bad.

Re:How Not to Sort by Average Rating (2009) by dredmorbius on Thursday July 09, 2015 @03:55PM (#9860292)
A hybrid variant: 1,000 points/month, 100/day (yes, higher than the monthly average). Exceeding either on starts reducing the weight of total votes.

The periods should probably be rolling averages and apply to weights for CURRENT votes. Since early voting activity has undue influence, often in the first few minutes of contents' existence, retroactively deflating month-old ratings doesn't do much.

The idea is to enable reasonable inputs for a time, then start washing them out.

The deflation factor might be applied more broadly across other indicators (IP blocks, etc.).

Or ratings factored for conformance with stated site moderation goals. See my longer top-level comment.

Re:How Not to Sort by Average Rating (2009) by joshuapants on Thursday July 09, 2015 @12:19PM (#9858830)
> Same thing applies to things like Yelp reviews. Maybe a user with close to a 5-star lifetime rating average should have his reviews "renormalized" to 3's because his standards are probably just lower than the guy with a 1-star average.

Another problem is the perception of star ratings. It seems like 5 (maybe 4 also) is the only "positive" rating for many people. Anything less and it might as well be a 1.

Re:How Not to Sort by Average Rating (2009) by bwy on Thursday July 09, 2015 @12:46PM (#9859045)
Of course, there's many areas that I didn't even mention. Another thing in the same vein for upvotes that I sometimes think about is, what is the meaning of upvoting? Does it mean, "I like this," or can it also mean "I think this submission should be higher?" Maybe I think too much but I've refrained from upvoting posts I like because I don't think they should be higher than their current position.
Re:How Not to Sort by Average Rating (2009) by travis_bickle on Thursday July 09, 2015 @02:11PM (#9859672)
Can the design tell the user the meaning of upvote("I like this" / "I think this submission should be higher")? Because the significance of upvote could be either of the two depending on the area. Also, then one can sort accordingly.
Re:How Not to Sort by Average Rating (2009) by jmilloy on Thursday July 09, 2015 @12:51PM (#9859095)
Criticker does this well. But as you say, I try not to waste time watching movies I don't think I will like, so there is a good reason my ratings are biased towards the high end.
Re:How Not to Sort by Average Rating (2009) by timon999 on Thursday July 09, 2015 @01:35PM (#9859414)
But this encourages donwnvoting, which is something you definitely do not want.
Re:How Not to Sort by Average Rating (2009) by FooNull on Thursday July 09, 2015 @03:44PM (#9860239)
Why do you not want downvoting? Isn't that desirable part of the opinion gathering?
Re:How Not to Sort by Average Rating (2009) by Cthulhu_ on Thursday July 09, 2015 @10:55AM (#9858174)
> Do you think this will hinder the workings of sites like reddit?

I do. The user will think "Well given that I only have X to spend, I'd better be careful what I vote for". That will lead to less votes being placed, and thus, reduced interaction / community activity.

It's the same reason you don't want to cap the amount of submissions or comments someone makes. A flood control (max X per Y time period) to stop bots and spammers, sure, but no hard cap - or an inhumanly high one at least, bearing in mind that some people are indeed inhuman when it comes to voting/commenting.

Re:How Not to Sort by Average Rating (2009) by squeaky-clean on Thursday July 09, 2015 @12:42PM (#9859020)
I agree that showing the user that they are expending some sort of resource by voting will cause them to act differently or be more conservative with it.

An idea I've had is handling this behind the scenes by weighing the votes accordingly. Let's say each user only has 1,000 "points" but infinite votes. Votes 1,000 times? Each vote == 1 point. Just vote once? That vote is worth 1000x as much as the other guy. Spam a million votes on everything, and each one is basically worthless.

Of course, handling it exactly like that is way too simple, it would need to be balanced more carefully than that. In that naive example, someone who reddits for 5 minutes per month and votes once has as much power as someone online 24/7. Users would also be able to figure out how the system works if scores are visible in some way.

The specific use I played with this for (but never got around to finishing the project) was for movie reviews. There are people who will rate 50% of moves 5-stars and 50% of movies 1-star, but never in between. And some people who are much more selective and may give a 5-star rating to only a few movies ever. A 5-star rating from both of these users should not be valued the same.

Re:How Not to Sort by Average Rating (2009) by brudgers on Thursday July 09, 2015 @01:52PM (#9859536)
If people are upvoting "the right things" then limiting votes is counter-productive. If people are upvoting cat-pictures, then the problem is that the site is not selecting the right group of users and better filters are needed so that the community has the desired structure.
Re:How Not to Sort by Average Rating (2009) by brudgers on Thursday July 09, 2015 @01:50PM (#9859515)
Beyond the concerns you raise, vote caps etc., inhibit problematic behaviors mechanistically rather than via better social structures. Limiting the number of posts a person can make per day is one way of addressing potential flame wars when the underlying problem is that flaming is deemed acceptable behavior [and consequently such sites attracts people who engage in flaming and encourage some people to post in ways that get all their digs in at once].

The alternative of building community norms is harder. Mechanism can support it, but success depends on the over-arching social structure.

Re:How Not to Sort by Average Rating (2009) by lmm on Thursday July 09, 2015 @12:24PM (#9858870)
> I do. The user will think "Well given that I only have X to spend, I'd better be careful what I vote for". That will lead to less votes being placed, and thus, reduced interaction / community activity.

It's worth pointing out that Slashdot has run successfully for decades with users getting limited moderation points.

Re:How Not to Sort by Average Rating (2009) by dredmorbius on Thursday July 09, 2015 @12:37PM (#9858968)
Slashdot's moderation applies to comments only, not stories, and leaves a great deal of good discussion on the floor.

And other issues: https://news.ycombinator.com/item?id=9854240

Re:How Not to Sort by Average Rating (2009) by billmalarky on Thursday July 09, 2015 @10:56AM (#9858180)
I think sites like reddit just want engagement, even over quality. It's been known forever that reddit's algorithm leaves much to be desired but it's too risky to change.

IE the following (since patched) bug: http://technotes.iangreenleaf.com/posts/2013-12-09-reddits-e...

Re:How Not to Sort by Average Rating (2009) by jmilloy on Thursday July 09, 2015 @01:06PM (#9859207)
I find this to be a strange comment. Quadratic voting is much better if we have to reach a consensus and want to protect minorities. So it has it's application (and is woefully under-used). But surely it's not always the best method. I think often you want the most popular comment to be at the top of the list, because it's what appeals to the most people.

Secondly, one of the big problems that quadratic voting addresses is when an ambivalent majority outweighs a minority simply because everyone votes. But not everyone has to vote on every comment on a site like reddit, so the ambivalent majority is already ignored.

Re:How Not to Sort by Average Rating (2009) by nzealand on Thursday July 09, 2015 @11:14AM (#9858288)
It might work, if there was a nominal cost to creating an account.

It would be better to take the recommendation engine of netflix. Just throw it on top of reddit and charge a fee for tailored recommendations. Now that is something I would pay money for.

Re:How Not to Sort by Average Rating (2009) by dredmorbius on Thursday July 09, 2015 @03:51PM (#9860267)
Gresham's Law.

Fundamentally: assessing quality of complex products, including information goods, is hard.