The Draft Garnaut Report is to be commended for commissioning a study Global temperature trends – Breusch and Vahid (BV) by two prominent Australian National University (ANU) econometricians to examine global temperature series. The approach they take to modeling temperature has a long history. See for example at RealClimate,
Rybski, and Koutsoyiannis. Their findings that the significance barely reaches the 95% level with these kinds of models is not inconsistent with any of them. Even if there are no new insights, independent statistical studies by experts outside the field help to build trust and confidence in a controversial issue.

However, the results of the study are no reason for ‘high fives’ among proponents of man-made global warming. Despite concluding that “the temperatures recorded in most of the last decade lie above the confidence level produced by any model that does not allow for a warming trend”, the study also reveals just how close the case for anthropogenic global warming is to a spurious regression on a random walk.

Below are some of the issues that have been raised about the paper on the various climate blogs.

1. Straw-man?

Garnaut asked, “Is there a warming trend in global temperature data in the past century?” The presence of a warming trend in the last century has been pretty much uncontested by skeptics. Skeptics dispute claims such as found in Rahmstorf et al. 2007 that trends are greater than estimated by the IPCC models (about 0.2C/decade). This has been picked up by all of the green advocates to support swift action, “Trends in global air temperature, and sea level and atmospheric carbon dioxide are all tracking at or above the rates projected by the IPCC … ” (Roger Jones, “Keep cool is the message”, Herald-Sun, 10 December 2007).

Garnaut poses a straw man when he asks: Is there a trend in temperatures? Nevertheless, BV concluded that: “It is difficult to be certain about trends when there is so much variation in the data and very high correlation from year to year.” This is because temperatures very closely resemble a random walk, which has the effect of letting the series wander far and wide, with diversions that can look like trends. This means that apparent correlation of warming with greenhouse gases in linear regressions could be spurious.

2. One-sided or two?

Using a more advanced statistical random walk model, the trend was reduced from a average of 0.7 degC/century to 0.6 degC/century. Moreover, the significance of the trend was reduced from 99% to the 95% level. In fact, the trend would not have been significant in two series if they had used a more conventional two-tailed test with t=1.96.

Below are the critical values for the one-sided and two-sided t tests.

Its clear that for a 1-tailed test all series are significant. But for a 2-tailed test with 1.96 as a critical value, 2 series are not significant. A 1-talied test has been used because the question was about significance of a warming trend, not significance of a trend, per se. Either could be argued. Gerard E. Dallal believes that 1-tailed tests should not be done anymore and are universally deprecated by statisticians. One-tailed tests make it easier to reject the null hypothesis when the alternative is true. If there is no difference between warming and cooling, the probability of getting a significant difference by this approach is 10%, not 5% as it should be. The chance of a spurious significant difference is doubled.

Advice from Statistical Notes BMJ is that two-sided tests should be used unless there is a very good reason for doing otherwise. One-sided tests should never be used simply as a device to make a conventionally non-significant difference significant.

3. Cherry-picking?

BV conclude that “Viewed from the perspective of 30 or 50 years ago, the temperatures recorded in most of the last decade lie above the confidence band produced by any model that does not allow for a warming trend”. This is based on a graphical procedure also known as a slide-and-eyeball test (see lucia’s blog for criticism of those) in which neither the models nor the procedure used is described in a replicable way. What is it, 30 or 50 years? The graph seems to indicate 50 years — but why 50 exactly? Peter Gallagher asks if this is a case of cherry picking a start point to ensure a result?

And what is it with this footnote that indicates “The confidence bands in these graphs are calculated using the “dynamic forecast” option in Eviews and they do not incorporate estimation uncertainty.”? From the figure above it is obvious that a slight increase in uncertainty contributed by the parameters and the start point would put all temperatures inside the confidence band.

4. Satellites?

Geoff Sherrington asks why only surface record data and not satellite data was used. Unlike surface data, the satellite data have a wide and consistent global coverage and show a lower warming trend than the surface data. Given the marginal significance of the warming trend in the surface data, these data would probably have tested non-significant.

5. Monthly?

At The Blackboard lucia is perplexed about why they chose to rely on averages of calendar years rather than monthly data. Using the monthly data would give more total data points, and also permit them to include the recent data. This is important to answer the question “Has the trend changed recently.” As it is, they’ve left out 5% of the data, and in particular, data that are important to the specific question being asked.

This is a roundup of some of the comments about BV on the climate blogs at the moment. There seems to be a consensus that Garnaut does not garner as much support from the ANU paper as he suggests:-) I am not making any accusations, but some of the choices of data and tests make a conventionally non-significant difference significant. Such choices need justification. Despite concluding that “the temperatures recorded in most of the last decade lie above the confidence level produced by any model that does not allow for a warming trend”, the study also reveals that the warming is statistically almost indistinguishable from a random walk. This shows just how weak the case is for CO2 caused global warming not being due to a spurious regression.