https://www.cochrane.org/news/statement-physical-interventio...
... which is what I just said: some people got mad at them because their review found no reliable evidence that masking worked (or rather, that mask mandates worked, but these are virtually the same thing).
The null hypothesis for any medical intervention is that it has no effect. You start from that and then try to prove your hypothesis that it does have an effect, which is what medical studies are for. If you can't prove something works then we fall back to the null and assume it doesn't. So that isn't a misleading or inaccurate interpretation of the results, though it would certainly have been politically convenient for the Cochrane organization if their reviewers could have supported the claims of public health authorities.
Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence.
Which I think is definitive.
No, they're not. Which is the heart of the disagreement. Something can be both individually beneficial and yet impossible to implement at scale. The review at best established the latter, but people predictably spread it far and wide to harass those going for the former.
No it's not the same thing, and that's the key point. If you tell people that masking doesn't work (which is false) then of course mask mandates won't work because adherence will be low. A self-fulfilling prophecy really.
Also health authorities told people masks were highly effective. That's what justified the mandates. So you can't argue mask mandates didn't work because people were told it wouldn't work.
Therefore there's no self fulfilling prophecy here. It didn't even matter what individuals thought anyway, we all had to wear masks.
Although Cochrane much prefers to use RCTs, people have run regressions over the data and there was no link between levels of mask wearing and infection rates. It sucks but it appears that masks just can't stop aerosolized virus, which spreads like a gas. They aren't designed to do that so it's no knock against the manufacturers, who in some cases explicitly warned people that their products would be useless for that purpose (https://pbs.twimg.com/media/EfNmzptXkAEg9Od?format=jpg&name=...).
This is not virtually the same thing. Comparing those two is wildly disingenuous and you know it.
Oh.
I quickly found this:
"The new scientific review on masks and Covid isn’t what you think" Kelsey Piper
https://www.vox.com/future-perfect/2023/2/22/23609499/masks-...
Based on the criticisms, I expect Cochrane will revisit this topic.
Progress isn't a straight line.
For example i might "explain" to you that clean code is about writing the least amount of code possible and you might start code golfing your production systems.
If you want to know what the paper says, read the paper. Journalists are not scientists, most of them do not have the necessary knowledge to understand academic papers, nor do they have an incentive for doing it well. They do have an incentive for generating clicks though, generally by twisting the truth to make things sound more interesting or provoking than they are.
For example, after parsing through the ad hominem attacks and nonsense in that article, their main point is that the Bangladesh study found masks to be effective.
Except that study is junk and all the reported effects were found to be a result of researcher bias [1].
Vox also misrepresents the Danish study, which is probably the best study to date we have on masking effectiveness.
> Progress isn't a straight line.
Yes, but truth is most likely to be found in whatever facts are orthogonal to vox' narrative.
[1] https://trialsjournal.biomedcentral.com/articles/10.1186/s13...
1. Only two of the studies included are actually about COVID and masks; it's misleading to say, "oh, this big meta-review of all the data shows that they masks didn't do anything against COVID in the pandemic" when at best exactly one study out of the 78 included says that.
2. Even those two studies weren't about mask use but about mask policy. They're interesting from a public health perspective but don't tell us anything about the actual mechanics of whether masks work or not, which seems to be how (some) people are taking it.
It also doesn't speak well to the Cochrane review if they're including a study that you yourself argue is junk!
https://en.wikipedia.org/wiki/Ad_hominem
> "Re-analysis on the statistical sampling biases of a mask promotion trial in Bangladesh: a statistical replication"
I'm not remotely qualified to have an opinion.
That said...
The open (public) process as well as the critics sharing their source code is just awesome.
https://trialsjournal.biomedcentral.com/articles/10.1186/s13...
https://github.com/mchikina/maskRCTnote
I share the reviewer's hope that authors of the original study will respond.
https://dailysceptic.org/2023/02/06/dr-carl-heneghan-intervi...
So, a Cochrane review is a study which synthesises all available studies – all that we can find or identity – on a particular topic. It follows a highly structured format and is always preceded by publication of a protocol. All this is to minimise the bias. Also, it is extensively transparent. In this case we are looking at about 300 pages of review. Now, the review called “Physical interventions to interrupt or reduce the spread of respiratory viruses” is called in code A122 for short and I will be using that acronym simply because it is just too long a title. So the protocol was first published in 2006 and then the first version was published in 2007, updated in 2009, 2010, 2011, and then 2020, so this 2023 is the fifth update of this review. And the reason why we update the reviews is they are soon out of date if we don’t do that, especially in some fast moving topics.
This update didn't change the conclusions from any of the prior reviews.
Because masks are so politicized there were numerous attacks on Cochrane this time around, though nobody cared in any of the previous rounds. The Cochrane authors are aware of all the criticisms, but there were no justifications found in any of them to alter the conclusions of the review or their procedures for doing them.
True. But as noted elsethread, Cochrane is not responsible for others misinterpreting the conclusions.
"Statement on 'Physical interventions to interrupt or reduce the spread of respiratory viruses' review" https://www.cochrane.org/news/statement-physical-interventio...
"The original Plain Language Summary for this review stated that 'We are uncertain whether wearing masks or N95/P2 respirators helps to slow the spread of respiratory viruses based on the studies we assessed.' This wording was open to misinterpretation, for which we apologize. While scientific evidence is never immune to misinterpretation, we take responsibility for not making the wording clearer from the outset. We are engaging with the review authors with the aim of updating the Plain Language Summary and abstract to make clear that the review looked at whether interventions to promote mask wearing help to slow the spread of respiratory viruses."
> masks are so politicized
Indeed.
These mistakes will happen from the original scientists, they will happen at the stage of editorial boards, they will happen at peer review, they will happen if external third parties start systematically reviewing every RCT.
So Cochran must similarly be scrutinized for their errors, because they will be making them as well.
And that's even before we get to the political factors outside of science misinterpreting complex data for their own purposes...
It's wicked hard just to get reproducible results (one facet of the replication crisis). Much less the challenges you list.
Confusion and miscommunication is the norm. Rising above that take Real Effort™.
One of my formative experiences was on a team trying to adopt the processes from the book Applying Use Cases. So simple. Really What could be more simple?
We had shared purpose. We all read the book (among others). We discussed. We all thought we were good to go.
And then the wheels fell off once real work started. Turns out we didn't agree. On anything. What is "the system"? What level of abstraction are we working at? What does this line (points at diagram) here mean?
Writing this now, experiencing PTSD flashbacks, I can confidently say I would have never succeeded as a scientist.
No, that was a misinterpretation of the review in the Covid-skeptic sphere. Cochrane have had to issue a statement to clarify : https://www.cochrane.org/news/statement-physical-interventio...
tl;dr : half of the people given masks in these studies didn't wear them consistently or at all, dragging efficacy results down.
Do you want to be the person going around policing mask wearing?
First, some laws probably don't have any positive impact.
Second, there's a difference between accurately summarizing trial results and extrapolating that to the impact of a new law. If there was a death penalty for not wearing masks, perhaps compliance would be better than in the trials and an effect would be shown. This doesn't mean that the trial analysis is wrong, you just can't draw a conclusion about the law from the trial data.
Third, laws have multiple purposes including Justice and Punishment. Some murderers might have zero chance of re-offending but we still want to punish them as a matter of Justice, not because it makes Society safer.
Maybe this is a ridiculous metaphor since public health is not what the justice system is trying to accomplish by imprisoning murderers.
But also, you'll be fine to go on red in the middle of night, where noone is around. Personal responsibility is unfortunately removed by centralised regulations and laws. Thats the problem.
[1] "personal responsibility" makes little sense as a moral catch-all when ignoring your own health can cause other people problems
How much of Netflix cause health problem for you? What about food? How much of junk food are you eating? Or maybe we should ban tabaco and alcohol. Or profesional sport. All of that is potential threat for your health. Being overprotective could damage your mental health... etc.
More regulations = less personal responsibilty.
I will make a comparison, tell me how you find it: "Dumping waste into the river shouldn't be illegal because you dump your waste in your toilet every day and it goes to the river."
Does it sound reasonable? Sure. But I haven't defined scope and I haven't defined effect and I haven't even defined what 'waste' is; I just made up two scenarios which could have similar outcomes and then called it day -- but if you actually care about what happens to the world you need to get a little more detailed than writing silly grandiose political statements and making non-sensical metaphors comparing indicting someone for murder with crafting public health policy.
I generally think laws should be codifications of societal norms. Which also implies that as societal norms change so should laws.
So even things such as murder which people do and we don't want should be codified as illegal. But even if nobody committed murder anymore it should still be illegal as its against societal norms.
I was actually surprised by the mouthwash outcomes as well. Almost no one really talked about mouthwash, but it looked to be useful in the study.
But it's efficacy will really only be decent while it's in your mouth. Once it gets diluted past a certain point, it's not going to be doing anything. You'd probably have similar results with vodka.
No, it wasn't. You should read the paper itself, instead of relying on (sadly) biased editorials about the paper. It literally says what the OP wrote:
> Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness (ILI)/COVID‐19 like illness compared to not wearing masks (risk ratio (RR) 0.95, 95% confidence interval (CI) 0.84 to 1.09; 9 trials, 276,917 participants; moderate‐certainty evidence. Wearing masks in the community probably makes little or no difference to the outcome of laboratory‐confirmed influenza/SARS‐CoV‐2 compared to not wearing masks (RR 1.01, 95% CI 0.72 to 1.42; 6 trials, 13,919 participants; moderate‐certainty evidence)
The only place they found any plausible signal was comparing N95 respirators against surgical masks, but the evidence was extremely weak:
> We pooled trials comparing N95/P2 respirators with medical/surgical masks (four in healthcare settings and one in a household setting). We are very uncertain on the effects of N95/P2 respirators compared with medical/surgical masks on the outcome of clinical respiratory illness (RR 0.70, 95% CI 0.45 to 1.10; 3 trials, 7779 participants; very low‐certainty evidence). N95/P2 respirators compared with medical/surgical masks may be effective for ILI (RR 0.82, 95% CI 0.66 to 1.03; 5 trials, 8407 participants; low‐certainty evidence). Evidence is limited by imprecision and heterogeneity for these subjective outcomes. The use of a N95/P2 respirators compared to medical/surgical masks probably makes little or no difference for the objective and more precise outcome of laboratory‐confirmed influenza infection (RR 1.10, 95% CI 0.90 to 1.34; 5 trials, 8407 participants; moderate‐certainty evidence).
The editorial you cited was a low point in the history of Cochrane, where they gave in to public outrage and attempted to cast doubt on their own data.
https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD...
Whether you think the editorial was them caving or not, they also issued it under their own name with the same weight as their other reviews, so they must have thought enough of it to do so.
Given that there's ample laboratory evidence of the filtering capacity of a good N95 or even a KN95 mask, and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test, I think I'll stick with the mask in future and I have no hesitation recommending winter masking to others who believe they are at risk of complications.
I've liked not being sick for the last three years.
" [Cochrane] found there was no reliable evidence that masks worked against COVID/"
And the editorial: "the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive"
How is "inconclusive" functionally different from "there was no reliable evidence?" Seriously, how do you justify this pedantry while ignoring and obfuscating the truth?
People do much evil by focusing on the wrong facts, the wrong stories, and the wrong lessons learned, while ignoring the right ones. That you are willing to focus on apparently frivolous pedantry while ignoring the fact that so many were forced to use masks without any high-quality scientific evidence that they actually did anything, including children, and all the lessons that should derive from this, is in my opinion, very representative of this type of evil.
The latter sounds like advertising and education about masks rather than wearing the masks themselves. ie telling people to wear masks made no difference in spread probably because people’s minds were already made up about masking.
I din’t see it making any conclusion about masking itself
Throughout the pandemic, my family and I have been careful about being around others. We have masked with N95 or KN95 masks when we couldn't stay home. I cut my beard for a better fit. We got vaccinated as we were able, and enrolled one of our children in a vaccine trial (he wound up in the control group). We have tested pretty often, and as best we know none of us has contracted the virus. I'm meeting my responsibilities to society around not spreading coronaviruses.
I'm also trying to meet my responsibilities to society around engaging with issues honestly. It's disheartening to see people who probably share my position on an issue behaving poorly. Pretending that downsides don't exist is more likely to lose you the argument than win it. Spouting abuse and insults means people stop listening. Please do better.
There are studies that show cheaply made surgical masks shed microplatic fibers which end up in the lungs of those who wear them over prolonged periods.
How will that effect us down the road? TBD.
"We found no reliable evidence that abstinence prevents teen pregnancy"
"We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"
The first is obviously wrong, and if the the second is true it would mean the government should look for other ways to prevent teen pregnancy, but it wouldn't mean that practicing abstinence as an individual doesn't work to prevent pregnancy.
In medicine you cannot distinguish. It is all about the intervention and not about some theoretical best-case scenario.
The intervention is to ask people to wear masks. People comply as they in real real life and then we measure the results. There was no reliable evidence that this made any noticeable difference.
Now you can change the intervention – instead of asking and mandating masks as we did, we could educate masks wearers more. Unfortunately we have no evidence that it helps.
Perhaps masking could help to an individual wearer? Alas, we didn't collect such evidence either.
Some studies are lab based. In those masks had some effect. But that's not how people use masks in real life, so they don't mean much.
I think saying "Using X is effective, but only if you actually use X" is obvious. The thing people want to know is "do masks stop the virus" which is an entirely different question from "How many people will wear masks", which is a different question from "What is the effectiveness of interventions to promote mask wearing"
Therefore the real question is how effective is the intervention. It will be (or should be) asked by people responsible with public health policies.
P.S. Cochrane group is not for giving scientific answers to individual people. Its main aim is to evaluate the evidence of different treatments and provide guidance to policy makers and healthcare authorities.
We have similar problems getting schizophrenics to take their meds and getting communities with high rates of open defecation to use toilets, but nobody suggests that we give up on antipsychotics or sanitation facilities.
Every other treatment in medicine including schizophrenia is tested how it works in practice. It is incurable disease and the treatments have many side-effects. Thus the question becomes not “does this medicine cure schizophrenia” but “does this treatment works better than placebo or another treatment?”. When studies are completed, we gather evidence by monitoring real life experience with this treatment.
Medicine is tested according to how it works when people actually take it. People participating in research studies who fail to take their medications (or their placebo for that matter) are kicked from the program and their data is typically discarded entirely.
In fact, often clinical trials are statistically analysed by intention-to-treat, including all people who have been randomised even if they later don't receive the treatment.
Per-protocol-analysis (including only people who follow the study protocol) can also be used but it is more prone to bias.
Besides, with masks it is not simply wearing or not wearing a mask. Even a very diligent mask wearers may wear it in a way that makes it less effective without being aware of that.
In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed. If the real life situation is that most people take medicine in a way that makes it ineffective and so much that the clinical trial cannot find significant effect, then he shouldn't prescribe it. It is just a waste of resources and giving people false hopes.
Fair, that said, intention-to-treat is more likely to greatly underestimate the efficacy of a treatment when non-adherence is expected to be high/isn't being monitored.
> In short, when the doctor prescribes a medicine it is important to understand the factors why the patient may not take the medicine as prescribed.
I agree, but the solution is to help the patient overcome those barriers not to throw out the medication. It's to give people the information and tools they need to follow the treatment. People who wear masks could be trained on how to properly fit them, take them on/off, store them, replace them, etc. The real life situation around masking included basically none of that. "Wear a mask" was basically all people were told.
It doesn't make sense to fault/dismiss masking if a large part of the population isn't wearing them because they were tricked into believing that masks don't work or that masks will actually make them sick, and another large part of the population wears them, but wasn't shown how to do it correctly.
It's important to be aware that those things are going on within the population, but the next step from there is still "educate the public" and not "abandon all efforts at masking" - at least not until a more accessible alternative which is also as effective as masking becomes available
If we still could not find reliable evidence that masks are effective, then the policy makers should be told that.
There is very little you can do to improve mask wearing technique. We certainly explained these things to doctors, it made no difference in results. If you want to make more controlled studies, you can do that. Don't hold your breath however.
No, we should not continue requiring wearing masks because you are only doing that out of hope. That's not how we do things in medicine. It would be unethical. There are many medicines that show effectiveness in the lab but fail in clinical trials. We don't demand for those medicines to be used until we find more effective alternative. Many unknown factors could cause ineffectiveness in clinical trials, we don't need to understand all of them, just the fact that the drug failed to demonstrate effectiveness and safety in real life settings.
>> a more accessible alternative which is also as effective as masking becomes available
The point is masking was not effective. It has not shown effectiveness anywhere in the real world.
https://jamanetwork.com/journals/jama/fullarticle/2776536
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7191274/
> > "We examined whether promoting abstinence prevents teen pregnancy and the results were inconclusive"
> The first is obviously wrong,
No. They're equivalent. They both mean "we looked, and we didn't find any confirming evidence." You're confusing "we found no reliable evidence of X" with "we found evidence of NOT X", which is different, and essentially never achievable in empirical studies (note: this is not an invitation to get side-tracked in pedantic debates about proving the null; I'm telling you how actual randomized controlled trials work, in real life.)
Proving a negative via statistics is ~impossible, so what you do instead is to look for significant differences in X, attributable solely or partially to the intervention. If you don't find such a difference (as was the case in the mask review), you say "we found no reliable evidence of X".
But when the Cochrane authors wrote "Wearing masks in the community probably makes little or no difference to the outcome of influenza‐like illness", they really did mean exactly what it sounds like -- the effect size in an aggregated pool of randomized controlled trials was statistically indistinguishable from zero. You can debate whether or not they looked for the right thing (X), you can debate whether or not adding another big randomized trial would help find X, and so on. But the plain-text interpretation is correct.
Data is data. Editorials are editorials. The fact that they're published on the same website doesn't change the data. If the Higgs boson was published in the same issue of Physics Letters B as another letter that claimed uncertainty of the result, would you treat them with equal weight?
> and having worked with an N95 respirator in tuberculosis control settings for 17 years and never converted my TB test
I mean...that's fine? Nobody is telling you what to believe or do. Most of what we do comes without evidence. But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.
> I've liked not being sick for the last three years.
I haven't worn masks and I haven't gotten sick either. Other than Covid -- which I got when we were all wearing masks.
"post hoc, ergo propter hoc."
> But let's be slightly rigorous thinkers for a moment: there's a fairly obvious difference between a fit-tested n95 mask in a laboratory setting, where there are lots of other interventions happening at the same time (negative pressure labs, hoods, etc.), and putting on a loose surgical mask on a bus. We should be able to talk about that rationally, and not resort to superstition.
No one's resorting to superstition. You're the one saying there's no value in an intervention that has empiric laboratory evidence to support it. The argument here is what matters at the population level. If the problem is performance, then we train people to select and use masks better, not simply say that there's no point to it at all.
Bear in mind a possible source of confusion here: TB bacterium are ~3 microns in size, but viruses are about 0.2 microns. The Cochrane review I mentioned is only about respiratory viruses. So it's possible that they may work against TB but not against flu or COVID.
The NIOSH definition for an N95 is a device able to filter at least 95% of airborne particles that have a mass median aerodynamic diameter of 0.3 micrometers. While SARS-CoV-2 is around 0.1 microns in size, naked COVID-19 viruses in air are rare as they would be torn up nearly immediately, so they are almost always within aerosols. Typical respiratory aerosol range is around half a micron or so [0], and as the aerosol particle size gets smaller, so necessarily must be the amount of virus that is present.
Is this perfect filtration? No, but no one gets sick from a single virus they inhaled either, even with as communicable as the current Omicron variants are. There's a minimum infective dose and they help keep exposure under it.
I've seen it directly--one woman putting on a mask when I approached. The thing is she had been hiking near the back of the pack in a group that got together for the hike. She was at a far higher risk from being downwind of her group (this was not a family bubble) than of me being off to the side.
I can basically guarantee nobody there was experiencing any appreciable symptoms (10,000' up, miles from the cars--not something you're doing with any sort of respiratory infection) but most Covid spread is presymptomatic.
A solo hiker masking when someone approaches makes sense (and is what I did pre-vaccine), but not masking with your group but masking for a stranger? That's merely an illusion of safety and why masks "don't work".
There's also the problem that the Cochrane data included mostly studies of things other than Covid--when you go over their own data only looking at Covid you do see some benefit. Note, also, the pooling of masks and respirators--we already know masks do little against the Omicron variants. Respirators or don't bother.
Cochrane messed up badly in this case by looking at the wrong thing. I'm reminded of the BMJ study showing zero safety benefit from parachutes when jumping from an airplane.
They found only mid-to-low quality evidence supporting the use of masks to prevent ILI. That evidence, for everything but the question of "n95 vs. other", showed an effect size statistically indistinguishable from zero.
You're essentially saying that the error bars on that effect size are big. They are. But they're still centered on zero.
> "There is uncertainty about the effects of face masks. The low to moderate certainty of evidence means our confidence in the effect estimate is limited, and that the true effect may be different from the observed estimate of the effect...There is a need for large, well‐designed RCTs addressing the effectiveness of many of these interventions in multiple settings and populations, as well as the impact of adherence on effectiveness, especially in those most at risk of ARIs."
OK, fine. If I grant this (I don't, but let's run with it...), it means the following is also true: we mandated something based on such low-quality evidence that no solid conclusions could be made from it.
For example, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8768005/ is "a detailed performance evaluation of the mask is studied from an engineering point of view," which aim to look at how the physics of N95 masks hold up against covid. What the physics shows is that N95 filtration helps block covid particles.
Cochrane (formerly known as the Cochrane Collaboration) is a British international charitable organisation formed to organise medical research findings to facilitate evidence-based choices about health interventions involving health professionals, patients and policy makers.[4][5] It includes 53 review groups that are based at research institutions worldwide. Cochrane has approximately 30,000 volunteer experts from around the world.[6]
The group conducts systematic reviews of health-care interventions and diagnostic tests and publishes them in the Cochrane Library.[7][4]
https://en.wikipedia.org/wiki/Cochrane_(organisation)