"Kneejerk" skepticism is a good thing. The real problem is posting kneejerk incredulous comments before bothering to read all there is to read about the matter, since reading further would often clear up such concerns.

This kind of skepticism isn't actually skepticism. Its internalizing good arguments without understanding them, instead using the phrases associated with these legitimate concerns as magic words for social dominance and denialism.

The typical "correlation doesn't imply causation" retort we see denialists level is a classic example of how people tend to use a phrase to avoid a reason-driven conversation about correlations (of which causal relationships are a subset) and instead shut down the entire topic.

The core of skepticism is inquiry, not denial. And modern Reactionary skepticism (because that is what it has evolved into now: a kind of reactionary culture clique) is all about stopping any inquiries it isn't predisposed to prefer.

I think the term for this is "thought terminating cliche." There may be some of that in this thread, but reading those questions/concerns charitably I think most people are curious, but perhaps a bit too busy/lazy to read the article in full before raising their concerns. The trope that being wrong on the internet is the best way to get somebody serve you the truth on a silver platter may also be in play, though I'd expect to a lesser degree. ("I wonder if they controlled for X" -> "I'll accuse them of not controlling for X, so that somebody will tell me whether they did.")

I don't really think it's productive to pathologise people as "denialists". People probably don't think of themselves using that term, so using it allows people to disassociate the critique from their own personal behavior. I think it's more useful to recognize that some tendency towards low-brow responses exists in all of us to some degree (particularly on sites with tempting point/karma/ranking systems), and we should all strive to do better.

Metagaming the purported hidden motives of people you disagree with is not rational. There is no evidence for your claims about how those you disagree with manipulate social discourse in order to shut down topics.

I'm not talking about intent. In fact, I dont know if they feel like they're being good skeptics or not. What I do know is people engaging in this fashion are not being skeptical no matter how they feel or what they intend to do.

I don't think most people are doing the thing you're implying they're doing. Nobody is trying to shut down the conversation. Most studies do an incredibly poor job of controlling for exogenous variables.

It's pretty rare that I read a paper that has not at least given some thought to the types of obvious correlation/causation concerns that come up after just reading the title of the paper. That said I'm always interested in what comes up in discussion threads here, so I wouldn't want to shut off the skepticism.

That itself is an apocryphal claim, but even if I take it at face value it doesn't really apply to curated data sets that we tend to be exposed to by media outlets with modestly competent science departments. The Economist is certainly not the best out there, but it's very far from an uncurated data set.

I would argue that this kneejerk skepticism and incredulousness are actually two sides of the same coin. Neither demonstrates actual curiosity or desire to learn.

I don't think there is any conflict between skepticism and curiosity. I'm sure most of us have been curious and skeptical of something at the same time. I find that skepticism compliments curiosity, since unbridled enthusiasm can easily corrupt my perception of reality while my desire to perceive reality is derived from my innate curiosity. I aim to never stop learning, and to always remain skeptical. Failing the later, I will fail the former.

I think you mean credulity, incredulous(ness) refers to a lack of belief and is thus synonymous with skepticism.

Well, I think it's a fact¹ that a large majority of "science" stories that go viral turn out to be wrong.

So the rational approach to them is "what might be wrong with this one?".

¹ Citation Missing

If you want to avoid this, just read the paper.

I don't understand why technical people waste their time complaining about this stuff. We all have the acumen to read, understand, and debate research.

> I don't understand why technical people waste their time complaining about this stuff. We all have the acumen to read, understand, and debate research.

But to be honest, the reason I read the comments (at least on a place like HN) is to get the key insight / failure without reading the article.

When scientists disagree among themselves, it's hard to use this strategy.

Reading scientific papers uses less time than complaining about them?

Is there a study to back that up? :)

It's paywalled. https://www.sciencedirect.com/science/article/abs/pii/S00950...

If I had access, I'd try to dig into more depth into how they approached decoupling this from temperature, because that seems like a really strong confounder. I'm also a bit curious about the underlying physical cause of the change in pollution levels, (weekends? summertime?) because that seems like it could also be a confounding common cause.

Alas, I'm not interested in paying $35 for the paper.

Preprint from the Colorado State website:

https://pierce.atmos.colostate.edu/Monthly_Pollution_and_Cri...

Thanks!

Just use SciHub like everyone else. Even if it's not up on SciHub the Supplementary MAterials are usually available free.

Because there’s only 24 hours a day? Because my tax dollars are often funding this research?

Why not take this farther and just do the research myself, eh?

from my experience, when science stories that go viral turn out to be wrong it's some secondhand reporting by people misunderstanding or misrepresenting the actual firsthand report

That's definitely also a factor.

The replication crisis says otherwise.

Both of those phenomenona are common. They aren't mutually exclusive.

I tend to agree, and this article is no different. Articles often report the underlying research in misleading ways.

The rational approach to all peer review is to assume there's a flaw and look for it. Quite a few commenters seem bothered by this reflexive reaction to published research, when the entire point of publishing research is to have it picked apart by critical thinkers everywhere.

For as long as journals refuse to publish negative results, researchers will scrounge for some type of positive result, even if it's causally questionable, because to do otherwise is to see years of work go for nought. Until these incentives change, the rest of us are right to be skeptical.

Skepticism is about inquiry, not denialism. You're not practicing skepticism if you make a decision that something is false. The point is to defer decision (or threshold your decision probabilisticly) on the evidence as it becomes available.

Saying "no" to everything is no different than saying "yes" to everything from a logical perspective.

> Saying "no" to everything is no different than saying "yes" to everything from a logical perspective.

This is incorrect. The base rate of true findings when talking about causal models is extremely low. If you say "no" to every published finding, you will be right much more often than you're wrong.

Now, that doesn't mean you should say no to every published finding. But the idea that "yes" and "no" should be equally weighted in your priors across the board is an inaccurate representation of the state of research, and the underlying facts of the universe.

I agree with this, but we're not talking about equal weighting here. We are talking about absolute bias is towards no or yes without any inquiry.

The actual prior distribution of effective to ineffective models is extremely hard to infer from everyday life. We don't have access to unbiased data sources. That's why we should focus on inquiry rather than canned responses.

As an example of how this can be misleading common the average person sees only a tiny fraction of the proposed models that are much more likely to be valid because they've passed far enough along the process of publishing to have received some scrutiny and some credibility in the average case.

I think we probably actually agree. I am also opposed to any sort of inquiry-free canned response. I do think that, if you don't have time or desire to investigate, your baseline should still be that results like this are probably false. Even given the bit of investigation i've done into this study, and even though I think it's decently well done and reasonable, if I had to bet money on it, i'd bet that it's false.

However, I would say that if you don't have the time/desire to investigate, your comments in public forums probably aren't worth listening to. So, I do agree that people that simply respond with "correlation != causality" without reading the study probably shouldn't be doing that.

> As an example of how this can be misleading common the average person sees only a tiny fraction of the proposed models that are much more likely to be valid because they've passed far enough along the process of publishing to have received some scrutiny and some credibility in the average case.

Ya, that's definitely true. The base rate of true models in published research is almost tautologically higher than the base rate of true models amongst all possible models. I was going to say that it is tautological...but I suppose it's actually not. All published models could be false. But I certainly agree that the research process is a pretty good filter, and the base rate of true models is almost certainly higher. But i'd be very very surprised if it was higher than 50%, even if you use a fairly high standard for "published research".

Researcher here (in ML), our field is so full of noisy results due to this issue. Everyone talks about it, but you can't get around the fact that you no longer can get away with a low amount of publications.

I dont see why journals can't publish more negative results. In math for example, its a big deal if someone disproves a hypothesis of someone famous.

The math equivalent of a negative result would be more like "I tried for months to prove this theorem, and it didn't work".

The difference is that in math "here is a proof" is all you need, while in sciences people are typically sharing what amounts to collections of observations. If you publish the collections that lead to interesting conclusions, but not the ones that are boring, people looking at what's been published are misled.

A negative result means simply they didn't find compelling evidence for their hypothesis. Nothing is proven or disproven (although it adds weight to the idea that the hypothesis is not true), so people think it is unexciting. For that and other reasons, the system incentives not publishing these, which is a pretty big problem.

My knee-jerk reaction to this was to a find a confounding variable.

Maybe people get more violent when the weather is hot, which is also when we use the most energy? I know cops always complain about hot days.

They controlled for temperature.

Sure it sounds plausible because it fits the world view of most here. The whole reason we have the scientific method and statistics is to remove personal bias that from the equation because if you don't you get nonscientific crap. The process only works if you use it every time. You can't just say "meh, good enough" because you find the results plausible. This study did not attempt to do a very extensive job controlling for other variables and likewise we can't just accept the conclusion at face value because we find it reasonable, more work needs to be done.

It is precisely the process you describe -- that people are very prone to thinking about something for "a few seconds" and "find[ing] some plausibility in the notion" -- that makes the rationalist scientific method important in the first place.

If we could just think about something momentarily to evaluate its probable truth, we would have no need for experiments, hypotheses, or sorting out correlation from causation at all. We need those things because superficial "think about it for a moment" analyses are very often totally wrong.

Also note that pointing out that this correlation does not imply causation is not at all the same thing as denying that there exists a causative link between the factors. It's simply stating that this evidence, in itself, is not sufficient to establish that. Other evidence well might.

>But really if the rationalists just took a few seconds to think about it

Even better, if they just took a few more seconds to actually read the article prior to commenting.

you literally just jumped to imputing a mechanism to back-support epidemiological data; I recommend not doing that.

What I did is propose a plausible hypothesis for further research.

Theory and speculation have a place in science.

What you did was link to a press release for a paper where they took 53 rats, removed their adrenal glands, injected them with corticosteroids, and saw they were more aggressive.

Few, if any, scientists would say that is a plausible hypothesis explaining the epidemiological results observed above. What you did was speculation with limited data and difficult transferrability (rats with adrenal glands are not representative of people exposed to air pollution).

Check out Robert Sapolsky's stuff on stress, disease and conflict in primates.

Then go and brush up on your research. Short term exposure to pollution is related to heightened inflammatory responses, which are in turn related to aggressive behavior... In humans and animals.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5874490/

There's much more where this came from.

I say "related" because we are talking about complex systems here. There are likely feedbacks involved.

hey I'm not disagreeing with exposure to pollution being related to aggressive behavior. What i'm saying is that there is no reason to believe that in this particular paper, that you can go from the epidemiological linkage results to a physical mechanism. Normally complex epidemiological observations are multifactorial, have multiple causes, and the associations are still only weakly correlated with those causal mechanisms.