1) Everything seems to work out perfectly, no matter how it is considered. For instance, if lead contamination in the ground is such a big deal, why does lead in gas track so closely to the crime rate? Wouldn't it be the case that it would track on the upside but not the downside?
2) He keeps on emphasizing lead lowers IQ. But the Flynn Effect suggests that IQ started going up in the 1930s and has only (maybe) started to level off very recently. It's the exact opposite of what you'd expect based on lead exposure from leaded gas.
3) Long term crime rates have been mostly going down for centuries: http://marginalrevolution.com/marginalrevolution/2011/06/lon...
Mind you, I'm fully in agreement that lead sucks and I sure as hell don't want my kid exposed to it. It's just that I'm far from convinced that the crime rates change mostly because of lead exposure. It seems far too tidy an explanation.
Part of me agrees with your overall conclusion about the corollary nature of the research as a whole, but those are my two cents on that particular point.
1) Maybe lead in the ground has the same general level of danger as lead paint? Other sources of lead were covered in the article.
2) It is possible (probable even) that the levels of lead have been retarding the growth of IQ, but not reversing it.
3) Ignoring that even your link mentions that the data is skewed due to higher survivability as medical technology has gotten better, this is a separate trend on a separate scale and not really all that relevant. It could possibly be explained by lead from other sources (plumbing maybe?) or other factors outside of the scope of this particular study. If anything, your link reinforces that there was an issue as the homicide rates climbed sharply before suddenly dropping off again.
Lead has a wider range of neurological effects than just reducing IQ. The causation may be more likely: lead -> unknown neurological changes -> more criminality & lower IQ.
And in case hackernews has become so infected with the kind of reddit thinking that can only see in black and white - yes I think reducing lead in gasoline is a very good thing.
In this case, lead exposure and crime rate are correlated, but maybe lead doesn't cause crime at all: maybe something else causes crime that also happens to correlate with lead exposure. Who knows, maybe a certain pesticide was used at the same time that lead gasoline came into vogue, and that's really the true cause of the rise in crime rates.
In research like this, when you can't do a manually controlled experiment, you have to control for hidden variables by some other means. And that's precisely what the investigators in this article did: they varied the input data to try to "shake out" other variables that might be behind the correlation. They looked at different time scales, different geographies, and different demographics, in an attempt to control for hidden variables that might be related to any one of those things. Every time you vary the input data and keep finding a correlation, your evidence of a causative relationship goes up.
The gold standard, of course, would be to expose two random, double-blind sets of infants to lead and to a control substance and see what happens. But since that would be unethical (as we have reason to think lead is bad for you), we're stuck with animal studies or retrospective studies. Personally, I find the evidence in this article impressive, but it would take quite a lot of looking into their specific methods to come to any real conclusion.
But in the case of lead, the causal chain from lead exposure to worse behaviour is fairly well understood.
The whole thing sound like classic fitting the preconceived conclusion. But there's also something like this:
>>>> Not only does lead promote apoptosis, or cell death, in the brain, but the element is also chemically similar to calcium.
How lead is chemically similar to calcium? They are on opposite ends of the periodic table and have very different chemical properties. That, of course, does not prevent lead from messing with ion channels, but saying lead is chemically similar to calcium is wrong.
How do you show that something is not caused by a hidden Z? All one can hope for is a preponderance of evidence. If the following passage from the article is true, then the preponderance here is stronger than a generic correlation-is-not-causation objection can just dismiss:
"In fact, use of leaded gasoline varied widely among states, and this gave Reyes the opening she needed. If childhood lead exposure really did produce criminal behavior in adults, you'd expect that in states where consumption of leaded gasoline declined slowly, crime would decline slowly too. Conversely, in states where it declined quickly, crime would decline quickly. And that's exactly what she found."
Thus your hidden Z would have to move in sync not only with X and Y, but also with their widely varying rates in different states. That's a high bar – especially if there are legal and institutional explanations for why leaded gas was phased out at different rates in those places.
For example, if you're doing an experiment, you should report everything that you think might make it invalid — not only what you think is right about it; other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked — to make sure the other fellow can tell they have been eliminated.
I did not see such explanation done in this case - it looks like everything described talks about what matches the hypothesis, and no effort is done to look for alternative explanations and refute them.
>>>> Thus your hidden Z would have to move in sync not only with X and Y, but also with their widely varying rates in different states.
Use of gasoline is highly correlated with use of cars, which again is highly correlated with income, population density, business activity, etc. Any of which can be also associated with crime. I.e. if both effects are caused, say, by population density (I do not say it is, I just take it as an example), then gasoline use would raise and crime would raise when population density raises (say, because some economic factors attract people, like it happens now in North Dakota due to shale gas boom) and both would fall when some factors cause population to move away - as it happening in Detroit, for example. There might be also other factors.
>>> In fact, use of leaded gasoline varied widely among states, and this gave Reyes the opening she needed. If childhood lead exposure really did produce criminal behavior in adults, you'd expect that in states where consumption of leaded gasoline declined slowly, crime would decline slowly too. Conversely, in states where it declined quickly, crime would decline quickly. And that's exactly what she found.
And this is exactly what she would find if both were strongly related to some third cause Z, but completely causally independent from each other. The fact that the MJ author takes it as evidence confirming the causation means they do not understand how causation needs to be proven. It is not evidence for causation, since it equally applies to both when causation exists and when it does not.
Yes, this means causation is hard to prove. It is.
Does that mean that causation is proved? Of course not. But it does mean that the evidence here is stronger than garden-variety correlation.
By doing a controled study.
Of course, in this case, we're better not knowing than paying that kind of cost.
How could the entire population ever be better off 'not knowing'? If there was a button to exchange a few babies or animals lives for the possible outcomes presented at the end of the article, I would do it in a heartbeat.
Then you go on to explain how lead is like calcium in its bioavailability, so in relation to the operations of a human body, lead is chemically similar to calcium (only dangerous).
http://brain.oxfordjournals.org/content/126/1/5.full
For example, lead’s ability to pass through the blood–brain barrier (BBB) is due in large part to its ability to substitute for calcium ions (Ca2+)
I's be interested in seeing what you think on that score after reading TFA and possibly even the studies that it talks about.
Honestly I don't know, but I do think there is a real problem in our understanding of how to interpret population level trends. The rigorous camp simply wants to throw out everything that isn't a double blind study. The optimistic are convinced of every similar looking graph. I think we should be doing more meta-analysis of these things and building up a Bayesian sort of set of priors to apply to these results.
> A second study [23] found that high exposure to lead during childhood was linked to a permanent loss of gray matter in the prefrontal cortex—a part of the brain associated with aggression control as well as what psychologists call "executive functions": emotional regulation, impulse control, attention, verbal reasoning, and mental flexibility.
Hate to drive by this interesting thread, but wasn't this same crime drop attributed to the legalization of abortion?
http://en.wikipedia.org/wiki/The_Impact_of_Legalized_Abortio...
Via http://election.princeton.edu/2012/12/22/scientific-american, which cited Reyes' work on the lead theory, here's a link to a striking critique of the abortion theory:
http://www.economist.com/node/5246700?story_id=5246700
Someone inspected Donohue and Levitt's code and found a bug that meant they hadn't controlled for what they claimed they had. "Fixing that error reduces the effect of abortion on arrests by about half, using the original data, and two-thirds using updated numbers."
What I really find to be problematic is a small time series of data of a facet of a human behaviour across a population is probably going to correlate extremely well to many other time series across the same population. Or even other populations. How unusual should we find the correlation?
What we might be seeing is nothing more than the average rate of change in a population to legislative changes. If it even shows that. If you take two time series that rise and then fall and overlay them on their peak I suspect most graphs look as convincing as the one presented.