A classic example I recall is a Feynmann story, where a group of researchers were getting very statistically sound and repeatable results of very unusual and unexplained particle track behavior in a cloud chamber. Feynmann looked at the data and said "you probably have a tiny piece of metal in the cloud chamber somewhere" and that turned out to be the explanation.
Similar examples in the social sciences include systematic bias in the preparation and administration of IQ tests to different groups of people (see Charles Murray's 'Bell Curve' vs. Stephen J. Gould's 'The Mismeasure of Man').
Hundreds of other examples can be found across all scientific disciplines, unfortunately. To quote the smartest PI I ever worked for "There's a lot of BS in statistical analysis".
It seems like there's this extreme reaction against people behaving like correlation equals causation, but instead of over-emphasizing correlation, it gets dismissed entirely.
Ruling out chance through significance and power, what phenomenon in the world is correlated without being causally related _somehow_?
Repeat every week with the worst traffic intersection for accidents.
What you find is. IT WORKS! HURRAH! These intersections are more than usually not the worst the following week! The evidence is clear. The correlation is utterly compelling. It is significant. It has power. How could it possibly be unrelated?
Now if we stop it being comically silly in our example and make it a red light & speed camera, see how the issue is much more difficult. There is a clear line of potential causation of fixing dangerous intersections. But is it really better than folk dancing for 20 minutes at 2am? [1]
[1] This example should not be interpreted as being an opposition to all red light and speed cameras.
So in other words, several of the commenters here are right. On the one hand we shouldn't jump to conclusions, but on the other hand we should listen to the clues.
Nevertheless, the fallacy is so commonplace. You will easily find a seemingly educated person selectively balking at the notion that causal relationship is ultimately a conjecture or at the notion that causal relationship is possible, depending on their pre-existing beliefs.
Being emotionally attached to purported causal relationship X->Y, they will count all correlational evidence in favor; when pointed out that the evidence is correlational, they will wave it off with more correlational evidence.
If that causal relationship does not happen to align with their world view, of course, they will be right onto you with the old correlation-does-not-imply-causation mantra.
There's billions of possible variables. There's N^2 possible pairs of variables. It's not feasible to look at every pair that is correlated.
Not really.
so, everything defying common sense should be reviewed, to find why and how exactly is this happening.
If we can get there, then we can talk about what we can do to improve things from there.
Yes, I agree, and this is precisely my point. My experience with introductory stats was a heavy focus on the technical details, when in fact what would be more effective is focusing on statistical logic.
I’m specifically thinking that something like Judea Pearl’s The Book of Why would be good to introduce early in stats education.
I don't know, most of the misinformation on the matter I've seen is just flat out wrong. Not misinterpreting statistics incorrect, just flat out lying, using false numbers or statements, etc. Having a cursory knowledge of statistics won't help you if you're incapable of Googling to check if a statistic is true or not.
I think a lot of it is more ideological at the "average" citizen level - on both sides. Most people aren't looking at the data, they're believing whatever they're told that aligns with their personal beliefs.
People intentionally or repeatedly inadvertently publishing BS should be called out and their opinions down-weighted heavily.
People having a good BS filter is one step in the chain towards this down-weighting, but isn’t the whole chain. If anything, people slinging BS today are likely to gain additional reach and opinion-weight, rather than to lose it.
But thinking about this from a loosely signal-process-y angle, once you have a BS signal in one channel that routes to a a particular set of minds, how does one route the corrective signal to that same set of minds (as opposed to, say, just your local friends who already agree with you anyway)? And even supposing you do, those minds are already programmed to process these signals in particular ways that probably means they're predisposed to accepting one or the other before either signal even arrives.
The answer to this has always been to address the root of the problem, the preprogramming, through (relatively) uniform mass education, because that's the one and only place where you can (relatively) uniformly cram ideas into everyone's brain before they scatter to the winds and begin fancying themselves "free thinkers".
Right now, sources of bad information get systemically louder, not quieter, over time. That’s way worse.
This works well naturally in small communities, but once the society becomes too large, BSers, liars, and scammers can just move on to a new gullible crowd.
So we got social media with review & reputation management systems, and these are now, of course, promptly gamed to the max. Moreover, this gaming is being done by the very people who should be most de-amplified, in order to amplify their BS, inlcuding everyone from just trolls to professional RUS dezinformatsiya shops (that'll gather a lot of downvotes).
So yes, a huge part of the solution is to de-amplify the crowd that spews Bs or deliberate lies.
Sure, freedom of speech is a right, but no one has to be required to amplify you on their platform - that takes away the freedom of speech of the platform owner (e.g., if HN were required to amplify everyone, then moderation would become effectively illegal).
Why does this view gain downvotes so frequently? IDK, but it seems to be mostly readers with no nuance who think that freedom of speech requires zero restrictions, so any shadow of moderation or restriction rankles them, and they are not articulate enough to state a reason, but can still hit the down-arrow.
I don't think your opinion is unpopular because it's wrong. It's unpopular because it's empirically impractical. People don't suffer a reputational hit for publishing BS, and saying "But they should" doesn't get us anywhere (as you yourself point out). How do you propose that we actually cause reputational hits for such people?
One way is to teach additional statistical literacy to the general public.
There are probably other proposals worth trying and I'd be interested in hearing your takes on what they are. (One that might be effective - though certainly controversial - is for the government to say that Alex Berenson is actively putting lives in danger with lies and that, if the untruth of his statements can be proven in court, he should be subject to criminal penalties, just like Elizabeth Holmes is on trial for lying about her blood tests. But that seems a lot less good for society as a whole than teaching statistical literacy.)
Elizabeth Holmes is on trial for lying about things where she had a specific legal obligation to tell the truth. I’d never heard of Alex Berenson before today, but I doubt his situation is one in which he’s obligated to tell the truth. That’s okay. Sunday school preachers aren’t either and we’re cool with that. We’re good with the Santa and Easter Bunny myths.
The same is also true of more or less the entire field of nutrition, none of which has yielded anything useful, and has resulted in governments themselves spreading misinformation:
https://sebastianrushworth.com/2021/11/27/is-saturated-fat-u...
Incidentally, Berenson has written a long rebuttal to this kind of thing which you can find here:
https://alexberenson.substack.com/p/the-english-data-on-vacc...
This is a bold claim and I'd be curious to see you back this up. By "lockdowns" - do you mean actual, genuine lockdowns / quarantines (mandatory stay-at-home orders, government-distributed emergency food packs and other essentials), or do you mean capacity restrictions etc. that get called "lockdowns" in the popular media?
Note that my straw-man proposal (which I'm not seriously endorsing) is not that it should be prosecutable to have been wrong. Plenty of startups try to build something, and it doesn't work; they don't get prosecuted like Holmes. Holmes is facing prosecution for fraud, for knowingly telling falsehoods. If your implication is that academic epidemiologists, COVID experts, and the entire field of nutrition are all fraudulent as opposed to merely just going through the usual course of science - which, to be clear, I see as mostly but not entirely impossible - then yes, I think we have a rather serious problem on our hands, which we need to figure out for the survival of humanity, and I'll repeat my comment above: I'm very interested in knowing what proposals you have for solving it.
https://apps.who.int/iris/bitstream/handle/10665/329438/9789...
"Quarantine of exposed individuals", "contact tracing", "border closures" and "internal travel restrictions" are "not recommended under any circumstances".
"Home quarantine of exposed individuals to reduce transmission is not recommended because there is no obvious rationale for this measure, and there would be considerable difficulties in implementing it"
And: "Border closures may be considered only by small island nations in severe pandemics and epidemics, but must be weighed against potentially serious economic consequences."
Also: "There is also a lack of evidence for the effectiveness of improved respiratory etiquette and the use of face masks in community settings during influenza epidemics and pandemics."
Finally: "This document will serve as a core component of WHO’s influenza prevention and control programme in community settings".
Consider that this was published in 2019. Less than a year later every single position had been inverted, and epidemiologists / public health experts were telling the world that "the science" didn't just recommend all these things but outright required them. The masks issue is especially neuralgic because at the start they correctly stated that there was no evidence mask mandates would have any effect, then flipped their position overnight and asserted anyone who thought otherwise was ignoring science.
As you correctly argue, there must be never be punishments for being wrong. However this is not a case of merely being wrong. No new evidence came to light to cause these flips in position, and in fact - as predicted - lockdowns, masks and so on have had no impact on the course of the pandemic. The models that were used to justify lockdowns were never validated against reality, and even contained severe coding errors: the people behind them have never admitted they did anything wrong here. Moreover the people who pushed hardest for lockdowns consistently claimed that there was "scientific consensus" behind what they were doing, even though just a year earlier the WHO itself had published a document stating that their recommendations should never be used in any circumstances! In other words, they were lying.
And that's before we get to the fact that the WHO has a long track record of going into mad panics over diseases that are extremely mild or may not even exist at all.
Der Spiegel, 2010, "Reconstruction of a Mass Hysteria: The Swine Flu Panic of 2009", https://www.spiegel.de/international/world/reconstruction-of...
Unherd, 2021, "Is the WHO inventing diseases" (in reference to gaming addiction disorder), https://unherd.com/2021/11/whats-china-got-against-gaming/
Any standard that allowed people to be imprisoned for spreading misinformation that was applied fairly would quite simply result in the mass imprisonment of public health officials, advisors and experts.
Hence my question about what you mean by "lockdown." This document actually argues in favor of the sorts of things called "lockdowns" in the media, and against quarantines - which was not my personal position, but I think it definitely does not back up the claim that lockdowns (in the media sense) do not work, either.
I would also note that right beside all of the things you quote is a column "Quality of evidence," and the recommendations saying that you should avoid home quarantines and border closures are based on evidence of "very low" quality. So I don't think that you or the WHO actually have evidence saying that these things don't work. The WHO just went with their gut feelings that they were bad ideas. Now we have facts, which supersede feelings. That is an appropriate, science-based reason to change your opinions.
(Also, SARS-COV-2 is not an influenza pandemic, it's a SARS pandemic, and this document is about influenza.)
Edit: jesus christ you were right
(because of pregnancy)
Maybe basic introductory statistics is more useful than basic introductory calculus though.
Should the general population be better at such things? Yes, of course.
However, in nearly every case such things reach a broader audience via mainstream media. And how many times have we seen those entries confuse correlation with causation? We've seen it so many times that it's safe to assume it's intentional. Surely, after each incident of such negligence a teacher or professor or math savvy citizen reaches out to correct them. Yet? Never a correction or retraction?? Never a spark of "we need to educate our journalists"?
Repeat something often enough and it becomes truth in the minds of the receivers. Toss in confirmation bias and echo chambers and even if your better educated the masses, the media and those "journalists" would mitigate public's understanding.
A public education system that creates a population conditioned to believe whatever any 'authority figure' says is also a system designed to create a population ripe for exploitation by advertisers. Similarly, the ideal authoritarian state desires a population that is generally ignorant and obedient, and that's what's been created in much of the United States.
A general population that has the tools and skills needed to independently analyze the claims of government authority figures and cable TV and Internet advertisers, that's not what an elitist-authoritarian system desires.
It's very sad to see people who completely lack these tools and skills attempting to do their own well-intentioned analysis, they're so easily manipulated by dishonest actors. They know enough to distrust 'authority figures', but not enough to conduct independent evaluations of claims. Such people have been sabotaged by the educational system.
Maybe combined with motivated reasoning.
Statistics like any math class was just another pointless and imposed game of symbol manipulation, for most. Not something that affects how they see the world or how they process disinformation (which disables rational thinking by appeals to emotion, so rational capabilities aren't necessarily even the issue).
Humans are naturally using statistical type reasoning all the time and are very good at it. But when it comes to things like in-group out-group consensus-forming conformity mechanisms, the whole point is they _overcome rationality_ for social cohesion. To follow rationality instead of the group, you have to leave the group, which is unthinkable, so your mind prevents you from thinking it long enough to change your mind, by emotional terminations.
Another solution might be convincing people to not pay attention to so many hot botton issues and not turn everything into a debate. A certain amount of detachment is probably more healthy for individuals and for the society.
It is important because if you flip a coin 20 times and get 20 tails, there is probably something fishy with that coin and you probably should bet tails. If it is 100 out of 100 (or even 95 out of 100), there is effectively zero chance that the coin is fair.
This can be modeled using Bayesian statistics. You start by assuming with a reasonably high probability that the coin is fair, then you revise your assumptions as you get more and more data.
The general idea is: if a coin flip lands on tails for too many times, you should probably bet on tails. You shouldn't try that in casinos: games are seriously checked for fairness (of course, the rules make it so that the house has an edge). But in an informal setting, it can get you a small advantage.
It's harder to cure a gambling addiction than just giving them math facts. They don't even accept the fact they're addicts. To accept it would feel bad so they don't have the bad feeling thoughts. This emotional stuff just doesn't yield to facts.
We should teach math literacy -- how to use math to understand the world we live in, e.g. some stats as you mentioned, basic finance (the glories of compound interest), sizing stuff, etc.
There's an absolute jungle of information we're presented with each day, and clearly many people would just straight up buy the vaccines-cause-death statistics uncritically.
There's no critical thinking without critical stats. How are the numbers made, what do they mean? Just about every field requires you to understand this, especially the social science fields where we're talking about some quite substantial issues like replication crisis. Things like economics as well, they're things everyone wants to understand but we don't give people the tools.
Somehow we have also missed out causality, when we've had the tools for a while. I reckon it's actually quite teachable though it currently feels like an advanced subject due to historical quirks.
Try swapping the graph labels with: Took vitamin D pills, Excercised 30mins a week, Got the vaccine.
Most would agree that all three points probably have some level of health benefit. Suddenly the cause vs correlation can be much more difficult for the average person to determine.
Do all three reduce the death risk? Do some of them just also trend with more health conscious individuals? Are any of the three just completely bogus?
At any rate, I'm certainly not an expert. But there seems to be some missing BS filter where people can recognize something as silly if the example is silly enough but not in the exact same logic fail for something that seems "more reasonable somehow".
----
0: https://www.spaghettimonster.org/wp-content/uploads/2007/10/...