News for hackers. Stuff that matters.
Re:Users post more falsehoods after others correct them: study by s17n on Tuesday May 25, 2021 @09:22PM (#27285018)
> The researchers observed that the accuracy of news sources the Twitter users retweeted promptly declined by roughly 1 percent in the next 24 hours after being corrected. Similarly, evaluating over 7,000 retweets with links to political content made by the Twitter accounts in the same 24 hours, the scholars found an upturn by over 1 percent in the partisan lean of content, and an increase of about 3 percent in the “toxicity” of the retweets, based on an analysis of the language being used.

Rule of thumb, you can disregard any social science paper where the effect size is single digits.

Re:Users post more falsehoods after others correct them: study by emsal on Tuesday May 25, 2021 @09:48PM (#27285185)
As imperfect measures as they are, statistical significance measures like the p-value are more epistemologically sound than any arbitrary rule of thumb-threshold on the size of the effect.
Re:Users post more falsehoods after others correct them: study by BeetleB on Wednesday May 26, 2021 @12:11AM (#27286224)
> statistical significance measures like the p-value are more epistemologically sound than any arbitrary rule of thumb-threshold on the size of the effect.

Keep in mind that the GP isn't saying the effect doesn't exist if it's in the single digits, but that it is inconclusive and/or insignificant. Insignificant in the human sense, not the statistical sense.

A 1% increase in this behavior? Irrelevant to almost everyone.

This, of course, is not even getting into the issue of the reproducibility crisis, much of which did rely on p-values. While I personally am happy to do p-tests, the skepticism of small effects is well founded. Were someone else to try to reproduce the effects and fail, the standard defense is that the results are sensitive to the methodology used. It's much easier to invoke that defense if your effect is 1% vs 20%.

Re:Users post more falsehoods after others correct them: study by unishark on Wednesday May 26, 2021 @01:36AM (#27286781)
P-values are based on a normally-distributed sampling distribution which presumes the samples are randomly-chosen. It's hard to see how random sampling can apply very well here.

Suppose we did a comparison between the p-values of biased researchers desperate to publish, versus a conservative heuristic that doesn't believe small effects. Particularly for social science experiments like this, which would you bet on being able to assess repeatability better?

Re:Users post more falsehoods after others correct them: study by asdfasgasdgasdg on Wednesday May 26, 2021 @12:08AM (#27286207)
Statistical significance is a good measure of how sure we are there is a correlation, or in this case with an RCT, causation. But any layperson can look at a 3% effect and conclude, yeah, that's probably not that big of a deal. Or not, depending upon your preferences! No judgement. It's not something that requires a degree to determine, just an assessment of one's own values and the effect size in that context.
Re:Users post more falsehoods after others correct them: study by s17n on Wednesday May 26, 2021 @02:23AM (#27287062)
Re:Users post more falsehoods after others correct them: study by wolverine876 on Tuesday May 25, 2021 @10:28PM (#27285465)
My rule of thumb is to disregard snarky know-it-all comments by random people on the Internet.
Re:Users post more falsehoods after others correct them: study by d4mi3n on Tuesday May 25, 2021 @10:47PM (#27285596)
Oddly enough, this comment seems to reinforce the findings we just read about.
Re:Users post more falsehoods after others correct them: study by scythe on Tuesday May 25, 2021 @11:14PM (#27285801)
But now you've created three Borromean competing claims.
Re:Users post more falsehoods after others correct them: study by d4mi3n on Thursday May 27, 2021 @02:03PM (#27306122)
I hadn't heard of Borromean Knots before you made this comment and got a great read out of it. Thanks!

For others following up on this thread, Borromean Knots seem to be an extension of psychoanalytic theory that tries to describe social interactions through an object oriented model that highlights the differences between symbolic, real, and imaginary claims:

1. https://en.wikipedia.org/wiki/Borromean_clinic

2. https://larvalsubjects.wordpress.com/2009/12/08/borromean-kn...

Re:Users post more falsehoods after others correct them: study by voldacar on Tuesday May 25, 2021 @11:30PM (#27285936)
russell's paradox: the hn comment
Re:Users post more falsehoods after others correct them: study by temp8964 on Tuesday May 25, 2021 @11:15PM (#27285808)
Your comment does not provide any useful information at all. A merely personal attack.
Re:Users post more falsehoods after others correct them: study by temp8964 on Tuesday May 25, 2021 @11:22PM (#27285867)
My guessing is that their measurement of partisan lean of content and toxicity of retweets may have measurement error way larger than 1% or 3%, not to mention other noises in the stastistical modeling.
Re:Users post more falsehoods after others correct them: study by ChrisLomont on Tuesday May 25, 2021 @09:35PM (#27285102)
Why would real effects either be 0 or >= 10%? What would cause such a discontinuity across every single effect?
Re:Users post more falsehoods after others correct them: study by Guvante on Tuesday May 25, 2021 @10:28PM (#27285468)
It isn't that simple. We want to know what percentage of tweets have property X given event Y. We can estimate that using a sample and error bars.

Note that the quotes specifically avoid this inference because the error bars would be silly. 500 million total tweets a day can't be talked about based on 7,000 hand categorized tweets.

But non staticians don't have any reference point for how powerful a predictor it is so a short hand is < 10% probably means in the error bars so less believable.

Re:Users post more falsehoods after others correct them: study by a1369209993 on Tuesday May 25, 2021 @11:15PM (#27285807)
Taking your question non-rhetorically: because the error bars are wider than 10%[eg, 0, which finds a floor of 4% for anything based on polls], so anything less than that is consistent with p-hacking/random noise[eg see 1], and this is pervasive enough[2] that social science papers should be assumed garbage until proven otherwise.

0: http://slatestarcodex.com/2013/04/12/noisy-poll-results-and-...

1: https://www.explainxkcd.com/wiki/index.php/882:_Significant

2: https://en.wikipedia.org/wiki/Replication_crisis

Re:Users post more falsehoods after others correct them: study by maze-le on Wednesday May 26, 2021 @01:51AM (#27286855)
It's not that no effects occur in this range, but if they do, they are not discernable from random fluctuations, noise and other statistical (non-)effects.
Re:Users post more falsehoods after others correct them: study by fuzxi on Wednesday May 26, 2021 @05:15AM (#27288117)
It can be real but unimportant. An extra drink every week might increase my liver cancer risk by 4%, but that's negligible if it's the difference between a 1 in 1 million chance and a 1 in 960,000 chance.