News for hackers. Stuff that matters.
Re:CRU's "very artificial correction for decline" is dead code by thras on Saturday November 28, 2009 @10:52AM (#965219)
Anyone searching for outright fraud in in these emails is going to be disappointed. The problem is not fraud, but rather scientists who believe that they are correct, and that their job is now not science but public relations.

That said, finding this was a big deal: http://camirror.wordpress.com/2009/11/26/new-the-deleted-dat...

Re:CRU's "very artificial correction for decline" is dead code by spoondan on Saturday November 28, 2009 @12:46PM (#965316)
The only thing that's a "big deal" about that is the laziness and mendacity of its author. The data he found was published in 1998 and 2000, a fact he mentions without irony while arguing that the data was "hidden".

The linked article also claims that the NOAA "deleted" the post-1960 data from a data set (briffa2001jgr3.txt) put together for an article that doesn't use these data. In other words, what really happened, is that Briffa and his collaborators created a data set only containing the data they used, and this derived data set was archived by NOAA. That's hardly nefarious or even the least bit worrisome.

Now to the heart of the issue: Why was the post-1960 not used after 1998/2000? Why do all of the e-mails talk about "hiding the decline" and the code talk about stopping in 1960 to "avoid the decline"? The answer, it turns out, is right in the very Briffa 1998 paper your link mentions (but fails to cite). It's title is Reduced sensitivity of recent tree growth to temperature at high northern latitudes and it's about how temperatures reconstructed using Briffa's MXD (maximum latewood density) data diverge from actual temperature readings after 1960. The implication of the word "reconstructed" is very important to understand: Briffa's MXD data don't show an actual decline in temperature. They show a decline in a particular measurement of tree-ring density that had, until 1960, been a good indicator of actual temperature. All that's going on here is that data that is known to be inaccurate is not being used.

It's no big deal.

Re:CRU's "very artificial correction for decline" is dead code by motters on Saturday November 28, 2009 @01:25PM (#965353)
How do we know that post 1960 tree ring data is bad, and needs to be adjusted, yet pre 1960 data is good? What happened to tree growth after 1960?
Re:CRU's "very artificial correction for decline" is dead code by spoondan on Saturday November 28, 2009 @04:06PM (#965510)
Compare the reconstructed temperatures to recorded instrumental temperatures and/or cross-check the tree ring reconstructions with reconstructions from other proxies.
Re:CRU's "very artificial correction for decline" is dead code by thras on Saturday November 28, 2009 @02:02PM (#965390)
It's even better if you read the Briffa letter to Nature that he cited: http://www.nature.com/nature/journal/v391/n6668/abs/391678a0...

Something completely unknown is making tree rings too small after 1960. And if you correlate your tree ring data to post-1960 temperatures it makes historical temperature look way too high.

Bizarro-Briffa sent a similar letter to Bizarro-Nature warning of an unknown divergence problem and saying not to correlate tree rings to pre-1960 temperatures because it makes historical temperature look way too low.

Re:CRU's "very artificial correction for decline" is dead code by spoondan on Saturday November 28, 2009 @04:31PM (#965534)
I don't know what you're trying to argue. Tree rings underestimate current, not historical, temperatures. This means that, yes, if you calibrate to current temperature, the past will look comparatively hotter. This is a causal relationship.
Re:CRU's "very artificial correction for decline" is dead code by waterlesscloud on Saturday November 28, 2009 @04:39PM (#965544)
At what other periods to tree rings not correctly correlate to temperature? How do we know the last 50 years is the only time this has happened?

Since we do not know why the temperature has not correctly correlated over the last 50s years, we can't really know when else that has been the case, can we?

So why do we rely on tree rings for any period?

They are proven to be unreliable over a signifcant period, and we don't know if or how often that's also true of the past.

Re:CRU's "very artificial correction for decline" is dead code by spoondan on Saturday November 28, 2009 @05:39PM (#965617)
You can cross-check historical reconstructions to assess their accuracy for given periods. I don't know how often this has been done and if other significant, unexplained, and unaccounted for divergences have been found. I haven't heard of any, but I'm not an expert. For complete answers to your questions, I'm afraid you may have to review the literature.
Re:CRU's "very artificial correction for decline" is dead code by foldr on Saturday November 28, 2009 @07:49PM (#965728)
If you're not an expert, please don't claim that it's feasible to cross-check this data against historical reconstructions -- you clearly do not know whether it is or not.
Re:CRU's "very artificial correction for decline" is dead code by smcq on Saturday November 28, 2009 @08:25PM (#965765)
I'm not an expert in climate change either, but I'm a journeyman in statistics and I support his claim that it can be correlated.
Re:CRU's "very artificial correction for decline" is dead code by foldr on Saturday November 28, 2009 @10:31PM (#965872)
How can you support that claim without having any relevant knowledge? I don't see that a general knowledge of statistics is particularly relevant here -- we're all well aware that things can, in general, be correlated.
Re:CRU's "very artificial correction for decline" is dead code by gnaritas on Sunday November 29, 2009 @12:59AM (#966056)
Scientists don't rely solely on tree ring data for historical temperatures, it's one of many methods used. Thus the tree ring data can be correlated against other measurements to know it's accurate for past time periods. This should be obvious.
Re:CRU's "very artificial correction for decline" is dead code by foldr on Sunday November 29, 2009 @01:06AM (#966060)
> This should be obvious.

Thus, either (a) it has already been done, or (b) it is more difficult than you would expect for non-obvious reasons. So what is your point? Have you even bothered to find out if anyone has done it or not?

Re:CRU's "very artificial correction for decline" is dead code by smcq on Sunday November 29, 2009 @03:15AM (#966116)
Fallacy of bringing absolutely nothing to the debate and asserting to the point of ad homonim that scientists would have skipped basic statistics on their data. Please see the post immediately above yours and go demonstrate that the method is flaw before continuing this debate.

The standard of evidence to show fault is yours. Assuming that papers published in freaking NATURE have sound statistics is a reasonable thing for a layman to do.

Re:CRU's "very artificial correction for decline" is dead code by foldr on Sunday November 29, 2009 @01:32PM (#966587)
>Fallacy of bringing absolutely nothing to the debate and asserting to the point of ad homonim that scientists would have skipped basic statistics on their data.

I thought that's what you were asserting. My assumption is that scientists have correlated tree ring data with other sources and found it to be unreliable. But of course, neither my nor your assumptions about what's possible and impossible in this area are worth a damn, because we __don't know what we're talking about__. That is my point.

Also, all this talk about fallacies, ad homonim, etc. etc., is very "internet". Especially since you only seem to have a vague grasp of what these words actually mean. Do you think we could maybe leave all that stuff out and have a serious discussion?

Re:CRU's "very artificial correction for decline" is dead code by smcq on Sunday November 29, 2009 @05:01PM (#966880)
Moving the goalpost.
Re:CRU's "very artificial correction for decline" is dead code by waterlesscloud on Sunday November 29, 2009 @01:24AM (#966072)
Can you direct me to papers correlating tree ring data to temperature data outside of the 100 year range from the mid 19th to the mid 20th cenutry? You indicate it's obvious, but I'd like to see the actual papers.
Re:CRU's "very artificial correction for decline" is dead code by smcq on Sunday November 29, 2009 @03:18AM (#966119)
If you're really this interested in it and really suspect an entire scientific field has engaged in completely terrible statistics, it's basic statistics and I suggest you publish a paper yourself.

That or you go look for the papers or the reason the papers have not been written.

This game is old. The standard of proof to demonstrate the method is flawed based on poor correlations is on your side. The method to do the correlation is public knowledge. The data to correlate is public knowledge. R is open source. Go forth and correlate.

Re:CRU's "very artificial correction for decline" is dead code by foldr on Sunday November 29, 2009 @01:39PM (#966597)
>If you're really this interested in it and really suspect an entire scientific field has engaged in completely terrible statistics

You're confused about something: the post you're replying to clearly wasn't accusing an entire scientific field of engaging in "completely terrible" statistics. The poster was just questioning whether the comparison you were suggesting was feasible given our current (rather imperfect) knowledge of temperature changes over the last few hundred years. It may not be a matter of just getting two sets of figures and comparing them.

Re:CRU's "very artificial correction for decline" is dead code by smcq on Sunday November 29, 2009 @05:01PM (#966881)
Moving the goalpost. God damn, this place just became reddit. Cheers. Enjoy the cesspool you created.
Re:CRU's "very artificial correction for decline" is dead code by foldr on Sunday November 29, 2009 @06:26PM (#967030)
I honestly don't understand what you mean. But if you think these discussions are games in which the aim is to score points (as your analogy would suggest) please do sod off back to reddit.
Re:CRU's "very artificial correction for decline" is dead code by smcq on Sunday November 29, 2009 @10:35PM (#967281)
You wouldn't. Moving the goalpost is a fallacy, though given your complete inability to do original research I doubt you could ever find that out on your own. This thread started because you asked for unreasonable evidence in response to a post that said "They are proven to be unreliable over a signifcant period, and we don't know if or how often that's also true of the past." (emphasis mine).

This is a clear claim that climate science researchers have not done basic statistics when presenting new models. This claim has been substantiated by nothing except you requesting more and more detailed analysis from non-experts, which you discount every time we provide it because you assert that you would only listen to an expert.

Ironically, you are actually failing to respect the authority of experts, because the discussion is fundamentally an accusation that experts are failing to do basic statistics. You are accusing us of not being experts so you can't listen to us in the defense of the experts you're attacking by proxy.

I seriously fucking doubt you understood any of this. But it doesn't matter, I won't return. I have no tolerance for people who argue by asking questions and petulantly demanding unreasonable proof while providing absolutely nothing of their own.

Thanks for ruining hacker news.

Re:CRU's "very artificial correction for decline" is dead code by foldr on Tuesday December 01, 2009 @12:12PM (#970111)
>You wouldn't. Moving the goalpost is a fallacy,

My problem here is that I'm not sure which metaphorical goalposts you are referring to. By the way, constantly talking in terms of "fallacies" (especially these informal ones, as opposed to actual fallacies of propositional or predicate logic) just makes you sound like a petulant 15 year old. (Ad hominem -- I'll do that one for you.) Don't refer me to a Wikipedia page on the informal fallacy of the month; address what I'm saying instead. Otherwise, you may be guilty of the fallacy of Missing the Goddamn Point Because You're Looking For Fallacies Instead of Reading What the Other Guy has Written.

Getting back to the issue at hand, I can only suggest that you (a) reread the thread and (b) calm down. I am not accusing any scientists of failing to do basic statistics, and neither was anyone else. I am merely expressing my awesome ignorance of all things related to climate science, and suggesting that perhaps some of the other posters here are about as ignorant of these issues as I am, but lacking a suitable degree of caution in their statements.

Good luck with leaving HN. Sadly, I very much doubt that you will find any discussion sites worthy of your superior intellect.

Re:CRU's "very artificial correction for decline" is dead code by Tichy on Sunday November 29, 2009 @07:52AM (#966292)
I think you are confusing politics and science here. Apparently some scientists study the correlation between tree ring growth and temperature. This is a perfectly valid research area. It is OK to publish articles about, to announce to the world that at some time the correlation went away.

Climate research is research. It is not some know-it alls claiming one thing or another. It is making up theories and testing and aligning them with available data. It is trying to get new sources for data. And so on. It is an ongoing process.

Science never KNOWS anything. It only assigns weights and likelihoods.

We don't know if all history of earth is just an illusion, because we could just be a computer simulation that started 10 seconds ago.

So we will never KNOW if tree ring data at all times before 1960 correlated with temperature. All we can do is to collect evidence, make up theories and compare them with reality.

Disclaimer: I am not a climate researcher myself.

Re:CRU's "very artificial correction for decline" is dead code by defen on Saturday November 28, 2009 @01:46PM (#965371)
Isn't it possible that thermometers got better after 1960, especially when measuring temperatures at "high northern latitudes"? This seems more likely than assuming that trees somehow got worse at measuring temperatures.
Re:CRU's "very artificial correction for decline" is dead code by spoondan on Saturday November 28, 2009 @05:09PM (#965584)
Not really. If the issue was thermometers getting more accurate, other proxy methods would show a similar divergence.

Trees are complex organisms whose growth relies on many factors in a complicated relationship. Just as a simple example, if you have a long, hot drought, do you expect tree growth to be more affected by the temperature or the lack of moisture? The scientists must control for these kinds of factors to isolate the temperature signal. But when you're dealing with the interactions of complex systems, new factors can emerge and the methods need to change to keep up.

Re:CRU's "very artificial correction for decline" is dead code by thras on Saturday November 28, 2009 @01:21PM (#965349)
No, you are misinformed. Further, your tone is unhelpful.

This was the first time that the data was available. As McIntyre points out in the top of his post, the decline was discussed in the 1998 and 2001 Briffa papers.

Of course, scientists could not actually evaluate this because the data was not released. But that's not the issue.

The issue is, as McIntyre also points out, that the IPCC 2001 graph (and later graphs) simply truncated the climate reconstruction data in 1960 and pasted the (more accurate) direct temperature station measurement data onto the graph after 1960.

While this was done it plain sight, it's still lousy scientific practice. Why? After all, shouldn't more accurate data be used? The problem is that comparison of temperature reconstructions with direct measurement tells you a lot about the accuracy of your reconstruction.

In this case it tells you a lot about the inaccuracy of the reconstruction and the sorts of signals that are missed.

I'm sorry, the paleoclimate reconstructions available in the literature are not convincing. They often use dodgy statistical techniques, mixing data gathering steps with data correlation steps to increase apparent agreement with the instrument record. There have been huge problems with releasing data and there are often bizarre choices made in proxy record selection.

Without good paleoclimate reconstructions, it's hard to know how far outside (or inside) the historical standard deviation for temperature we are. I happen to think it's an important question to have answered. A trillion dollar question.

Re:CRU's "very artificial correction for decline" is dead code by smcq on Saturday November 28, 2009 @08:30PM (#965769)
You seem completely confused by the scientific method. They do not have an obligation to release all data, but rather to release the method by which data can be acquired. They released data for the paper actually published to justify their statistical analysis.

Any data that you receive from another scientist with the intention of reproducing their experiment is fundamentally tainted and violates the scientific method.

Please don't argue against straw men, it's annoying.

Re:CRU's "very artificial correction for decline" is dead code by yummyfajitas on Sunday November 29, 2009 @10:06AM (#966373)
It's you who is confused.

Scientists are normally obligated to release all data. It's the standard in most other fields (e.g., biology, econometrics).

Running the same data analysis on existing data doesn't count as reproducing the experiment. Running a different analysis on the same data is, however, doing new science.

In contrast, failing to release the data does count as "violating the scientific method".

Re:CRU's "very artificial correction for decline" is dead code by smcq on Sunday November 29, 2009 @05:00PM (#966878)
No. You are wrong.