Re:'Troll hunting' algorithm could make web a better place
by im3w1l on Saturday April 18, 2015 @06:34AM
(#9398929)
I think high predictive accuracy is not the only important consideration. For science it may be enough, but not for real world usage.
Because once you implement the classifier you affect the world. And you have to take into account what those effects are. Two properties I think are desirable: 1. It should encourage good behaviour. By this I mean that if you adapt to the get a better evaluation that means you also become a better member of the forum. This relates to not being gameable. 2. It should give everyone a chance. For example I could see how being poor could correlate with low quality posts. But shutting out all poor people means you can lose valuable perspectives, so its not an ideal solution. As long as your posts are good you should be welcome, even if your a pleb.
I light of these, consider what filtering for poor spelling actually does. What do we know that correlates with poor spelling? a) being a foreigner. b) being poor / uneducated. c) being underage. Those are the people you filter out. This goes against the 2'nd principle I mentioned of giving everyone a chance. It's a tradeoff, of course, and I could see how it's worth it sometimes.
Because once you implement the classifier you affect the world. And you have to take into account what those effects are. Two properties I think are desirable: 1. It should encourage good behaviour. By this I mean that if you adapt to the get a better evaluation that means you also become a better member of the forum. This relates to not being gameable. 2. It should give everyone a chance. For example I could see how being poor could correlate with low quality posts. But shutting out all poor people means you can lose valuable perspectives, so its not an ideal solution. As long as your posts are good you should be welcome, even if your a pleb.
I light of these, consider what filtering for poor spelling actually does. What do we know that correlates with poor spelling? a) being a foreigner. b) being poor / uneducated. c) being underage. Those are the people you filter out. This goes against the 2'nd principle I mentioned of giving everyone a chance. It's a tradeoff, of course, and I could see how it's worth it sometimes.