Shaviv and Miskolczi

Nir’s 2005 paper “On climate response to changes in the cosmic ray flux and radiative budget”, available as pdf here, provides a solid case linking cosmic ray flux (CRF) variations to global climate change. The effect is consistent over hugely different timescales, using completely different indicators — from cosmic sources of CRF at the Phanerozoic, to the shortest time scale of the 11-yr solar cycle. The fit is extraordinary. The statistics competent. The bottom line?

Thus, anthropogenic sources alone contributed to a warming of 0.14 ± 0.36 K since the beginning of the 20th Century. Using our estimate, we find Tsolar = 0.47 ± 0.19 K. We therefore find that the combined solar and anthropogenic sources were responsible for an increase of 0.61 ± 0.42 K. This should be compared with the observed 0.57 ± 0.17 K increase in global surface temperature [IPCC, 2001].

In other words, changes in solar forcing, amplified by changes in cloud albedo due to CRF variations, account for a whopping 80% of the temperature increase seen since 1900. The rest, 20%, can be attributed to AGW.

I thought it would be interesting to see if Shaviv’s theory fits together with Miskolczi’s. Miskolczi’s is a theory of constant optical IR depth for the atmosphere, a consequence being that fluctuations in climate mostly come about through changes in solar forcing, i.e. short wave SW in, not effects of IR absorbers on long wave out. Even if you are not happy with all aspects of his theory, I want to look at it from the point of view of a theory explaining why IR absorbers like CO2 might not have as a strong an effect on global temperature as the IPCC scientists assert.

Shaviv explains that the CRF variations bring about changes in the % cover of low altitude clouds (LACC), changing the optical depth for both IR and SW. However, the effects are not equal.

This result is also reasonable considering that the total long wavelength heating effect of LACC was estimated to be 3.5Wm−2 [Hartmann et al., 1992], while cloud albedo is responsible for a globally averaged cooling of 20Wm−2, implying that changes in albedo will likely be more important for changing the radiative budget arising from LACC variations.

That is, by far the greatest effect of variations in LACC are on the SW, reflecting more sunlight away when cloud cover increases, thus shading and reducing the solar isolation at the surface. So while Miskolczi says global warming can’t be due to long wave variation, because the system optimizes for stability at that wavelength, Shaviv says there is a variation in solar input, amplified 5 to 7 times by cloud cover, and it explains temperature variation far better than GHG’s across all known time scales.

The two theories are both consistent and complementary, providing the strongest basis so far, for a natural and not human explanation for global warming.

Nir Shaviv

The theory of this Israeli astrophysicist has gained traction as the great white hope of climate skeptics. Below are some sources of background reading.

Shaviv champions the solar-wind modulated cosmic ray flux (CRF) hypothesis, which was suggested by Ney, discussed by Dickenson, and furthered by Svensmark (see CO2 Science). Evidence consistes of correlations between CRF variations and cloud cover, correlations between non-solar CRF variations and temperature over geological timescales, as well as experimental results showing that the formation of small condensation nuclei could be bottlenecked by the number density of atmospheric ions.

Basically, high CRF ionizes particles that seed more clouds, causing cooling. Low CRF produces brighter cloud free condition, resulting in warming.

Recently, he reports in GRL that three independent data sets show that the oceans absorb and emit an order of magnitude more heat than could be expected from just the variations in the total solar irradiance, implying the existence of an amplification mechanism. Shaviv, says this predicts the correct radiation imbalance observed in the cloud cover variations that are needed to produce the magnitude of the net heat flux into the oceans associated with the 11-year solar cycle.

The Reference Frame had an article about Shaviv recently too, noting significant pushback by RealClimate, proof the CRF theory is a viable alternative to the GHG warming as the main explanation for recent warmth.

By the way, despite all the huge pro-greenhouse bias in the journals and elsewhere, the Shaviv-Veizer paper has 91 citations right now, while the almost immediate alarmist reply by 11 authors, including RealClimate’s Rahmstorf, Archer, and Schmidt, only has 24 citations.

A very instructive exchange ensued in May 2006 at the RealClimate post “Thankyou for Emitting” where Shaviv challenged masterfully (starting at post 37), until the team eventually threw in the towel around post 125.

On the subject of Rahmstorf, Shaviv’s own blog site ScienceBits refers to RealClimate as WishfulClimate.org in a post More slurs from RealClimate. He pins them as bleeding hearts and intellectual lightweights as well.

Realclimate.org continues with its same line of attack. Wishfulclimate.org writers try again and again to concoct what appears to be deep critiques against skeptic arguments, but end up doing a very shallow job. All in the name of saving the world. How gallant of them.

Since there is no evidence which proves that 20th century warming is human in origin, the only logically possible way to convict humanity is to prove that there is no alternative explanation to the warming (e.g., see here). My motivation (as is the motivation of my serious colleagues) is simply to do the science as good as I can.

But Nir is not an extremist discounting all effects of greenhouse gasses.

In fact, my best estimate for climate sensitivity implies that anthropogenic radiative forcing explain about 1/3 of the 20th century warming, in particular over the past few decades.

Some of the flavor of the debate between them can be seen from the following two comments at Shaviv’s blog:

Rasmus: You are wrong about the motivation about our critisism, Shaviv; we are primarily interested in doing good sicence. We want to unravel the facts behind climate variability. In science, one challenge other views if one finds them strange or not credible. This is what we habve done. You make claims based on your own subjective belief og based on far-fetched speculations. The fact is that the claim that the recent global warming is due to GCR is not supported be any real evidence; there is no credible trend in the solar activity or GCR in the last ~50 years.

Shaviv: Perhaps you’re right. But if so, then it means you should have the integrity to add at the end of your post (and not buried in the discussion below), an addendum saying that this particular critique turned out to be wrong, as Kranz et al. is not applicable to the Milky Way. I for my part would add a similar addendum to my response, specifying that my comments about motives was wrong.

Second, over all, there was a large increase in the solar activity over the 20th century, even if you discard the Yakutsk data (used in the Ahluwalia plot), and this increase explains a large fraction of the 20th century temperature increase if the CRF/climate link is real. As for the temperature increase over the 1990’s, see my response above. Some of the warming is due to the fact that although there was a decrease in the indirect solar forcing over the last cycle, it is still notably above the current forcing/temperature equilibrium (and therefore causes warming), and of course, some of the warming is anthropogenic.

The scientific issues are not settled.

And now, the rest of the story.

Dr Roy Spencer, has weighted in on Dessler et al 2008. Water-vapor climate feedback inferred from climate fluctuations, 2003-2008,

Whereas Dessler closes his paper firmly in the climate liberal camp.

[23] The existence of a strong and positive water-vapor feedback means that projected business-as-usual greenhouse gas emissions over the next century are virtually guaranteed to produce warming of several degrees Celsius. The only way that will not happen is if a strong, negative, and currently unknown feedback is discovered somewhere
in our climate system.

Climate conservative Spencer continues where he left off:

The Rest of the Story: Shortwave Feedback

The other half of the feedback story which Dessler et al did not address is the reflected solar component. This feedback is mostly controlled by changes in low cloud cover with warming. The IPCC admits that feedbacks associated with low clouds are the most uncertain of all feedbacks, with positive or negative feedback possible…although most, if not all, IPCC models currently have positive SW feedbacks.

But I found from the CERES data a strongly negative SW feedback during 2002-2007. When added to the LW feedback, this resulted in a total (SW+LW) feedback that is strongly negative.

Is my work published? No…at least not yet…although I have tried. Apparently it disagrees too much with the IPCC party line to be readily acceptable. My finding of negative SW feedback of around 5 W m-2 K-1 from real radiation budget data (the CERES instrument on Aqua) is apparently inadmissible as evidence.

In contrast, Dessler et al.’s finding of positive LW feedback inferred indirectly from the AIRS instrument, even though it is only 1.3 W m-2 K-1 (3.3 Planck response minus their reported 2.0 for the LW feedback parameter) is not only admissible, but the reviewers even let the authors call it “strongly positive” feedback. Sheesh.

The last calculations regarding the Planck response seem to suggest that the null value — no feedback response — should be 0.7. Is this right? This would impact on the determination of the significance of the result considerably.

Ronald Reagan's Birthday

Ronald Reagan was born on the 6th of February 1911. If ever we were in need of wisdom from the man who changed the free world, its now. Below are some of my favorite Reagan quotes, relating to some of the madness going on around us.

Soros Says Crisis Marks End of Free-Market Model That Started Under Reagan

Government’s view of the economy could be summed up in a few short phrases: If it moves, tax it. If it keeps moving, regulate it. And if it stops moving, subsidize it.

Obama Plans to Reduce Budget Deficit to $533 Billion by End of First Term

Governments tend not to solve problems, only to rearrange them.

I am not worried about the deficit. It is big enough to take care of itself.

Madoff Left No Sign of Thousands of Reported Client Trades, Trustee Says

I know in my heart that man is good. That what is right will always eventually triumph. And there’s purpose and worth to each and every life.

Microsoft, Intel Firings Stir Resentment Over Visas for Foreign Workers

Recession is when a neighbor loses his job. Depression is when you lose yours.

My philosophy of life is that if we make up our mind what we are going to make of our lives, then work hard toward that goal, we never lose – somehow we win out.

North Pole Explorers’ Arduous Trek to Prove Arctic Melt Speed

Trust, but verify.

Propagation of Uncertainty through Dessler

The following is an approximate propagation of uncertainty through Dessler et als. equation for estimating the strength of water vapor feedback λ. We have been looking at the error-bars in his recent paper Water-vapor climate feedback inferred from climate fluctuations, 2003-2008, not calculated in the published paper. Assumptions made are noted. Refer to wiki for propagation of error equations.

Here R is the top of atmosphere IR, q is the specific humidity and T is the temperature.

1. $$\lambda = \Sigma \frac{\partial R}{\partial q}\frac{\Delta q}{\Delta T} =K\frac{\Delta q}{\Delta T} $$

Rolling up the summation over the earths surface into K.

2. $$\lambda = K\frac{q_1-q_0 \pm \sqrt{2}\sigma_q}{T_1-T_0 \pm \sqrt{2}\sigma_T} $$

Substituting the values two endpoint years used in calculating the differences, and their uncertainties, using propagation of errors for differences, assuming independence.

3. $$(\frac{\sigma_\lambda}{\lambda})^2 = (\frac{\sqrt{2}\sigma_q}{\Delta q})^2 + (\frac{\sqrt{2}\sigma_T}{\Delta T})^2 $$

Substituting uncertainty of q and T into equation for propagation of errors through ratios, assuming independence.

4. $${\sigma_\lambda}^2 = 2{\sigma_q}^2 + 2{\sigma_T}^2 $$

Assuming λ, q and T are the same magnitude. This is an underestimate if λ=2.

5. $${\sigma_\lambda} = 2\sigma_{qT} $$

Assuming uncertainty of q and T are equal, and squaring.

So according to these rough calculations, the actual uncertainty in λ could be roughly twice the uncertainty observed in the Dressler figures. This increase is due to the use of a single year, 2008 as the reference point, for calculating the change in humidity and temperature relative to other years. The uncertainty in the arbitrary choice of this point increases the uncertainty when propagated through the calculations for water vapor feedback.

Our calculated standard deviation of the mean was 0.37 W/m2/K. The confidence limits of the mean are then 1.96*2*0.37 or 1.45, giving a lower limits to the estimated 2.04 W/m2/K value of vapor feedback of 0.59 W/m2/K.

If we substitute values into step 3 of λ=2, q=2, T=1 we get an even higher uncertainty reflecting the effect on the ratio of dividing by a smaller number.

6. $${\sigma_\lambda}^2 = 2{\sigma_q}^2 + 8{\sigma_T}^2 $$

7. $${\sigma_\lambda} = \sqrt{10}\sigma_{qT} $$

The confidence limits of the mean are then 1.96*3.16*0.37 or 2.29, giving a lower limits to the estimated 2.04 W/m2/K value of vapor feedback of -0.25 W/m2/K. Being less than zero, this indicates that zero feedback is within the limits of uncertainty. This is very similar to the CI obtained be a t-test of difference of means in the previous post.

Dessler rambles on about the large influence temperature has on the uncertainty of the feedback here.

[20] Figure 4 also helps explain the large year-to-year
variability in our calculated values of lq in Table 1. It is
tropical q that primarily determines the size of the water
vapor feedback, and tropical q is primarily regulated by the
tropical surface temperature [e.g., Minschwaner and
Dessler, 2004]. The definition of lq, however, uses
changes in global-average surface temperature. While
changes in global and tropical temperatures are related,
there are often variations in the global average that are
not reflected in the tropical average and vice versa. Such
variations lead to large variations in lq.

[21] Consider, for example, the small feedback lq
inferred between January 2007 and January 2008. The
difference in the global average surface temperature DTs
between these two months was 0.60 K. Much of this,
however, was due to extreme changes in the northern
hemisphere mid- and high latitudes. The tropical average
surface temperature difference DTtropics was a milder 0.37 K.
The relatively small change in tropical surface temperature
leads to a relatively small change in q, and therefore a
relatively small value of (@R/@q)Dq of 0.57 W/m2.
Dividing that by the large DTs leads to the small value
of 0.94 W/m2/K inferred for lq over that period.

[22] The months with the largest inferred values of lq, on
the other hand, are the months where DTs is smaller than
DTtropics. For example, DTs between January 2008 and
January of 2006 was 0.28 K, while DTtropics between these
months was 0.33 K. This arrangement contributes to a large
value for the inferred lq between these months. Given
enough data, such variations should average out. In a short
data set such as the one analyzed here, however, such
variations can be significant.

You got that right. It would seem that three almost equal contributions to overall uncertainty are as follows:

Total uncertainty = measurements + reference point + ratio amplification

Dessler, Zhang and Yang fail significance tests

A concerned reader sent me this recent paper Water-vapor climate feedback inferred from climate fluctuations, 2003-2008, writing:

The following (ala Hansen) IMO should never have been accepted in a "peer reviewed" journal. "The existence of a strong and positive water-vapor feedback means that projected business-as-usual greenhouse gas emissions over the next century are virtually guaranteed to produce warming of several degrees Celsius. The only way that will not happen is if a strong, negative, and currently unknown feedback is discovered somewhere in our climate system."

Continue reading Dessler, Zhang and Yang fail significance tests

Productivity of Google Reader

The Google Reader tool is quite the step forward in productivity. Here are some of its most productive features.

Subscribe: to blogs, and create a single goto place to read blogs in a consistent format. Quickly scanning the blog posts in plain text is easier and quicker, as there are no ads or popups that get loaded if you go to the main website.

All Items: For someone like me who scans upwards of 20 blogs a day, the All Items tag lists all new posts from all subscribed blogs. You know which blogs have new posts, and a short summary to tell you if they are interesting enough to read. Save a lot of time going to each site.

Subscribe to Comments: Many blogs have a feature to subscribe to new comments, as well as new posts, so you can stay up to date with the conversation.

The Google Reader has entirely replaced my email subscription list, considerably uncluttering my inbox. Highly recommended.

Biased Research Studies

Detecting bias in research is not so difficult when you know what to look for. The conclusions are not justified by the data. Instead, the data may confirm, be consistent with, (or not inconsistent with) the conclusions. Working against this however are basic human motives on the part of the writer, to find novel and interesting approaches, find significant results when nothing is there, to be accepted by their colleagues, to get grants and be published.

According to Geoffrey Miller (The Mating Mind: How Sexual Choice Shaped the Evolution of Human Nature)

Geoffrey Miller: I think the interesting thing about human intelligence and capacities for abstract reasoning, and metaphor and analogy, is how very poor most people are at being evidenced based and sceptical. What we love to do is pick up little factoids and half-understood theories and repeat them to others to be interesting. Particularly on first dates. So we try to be interesting, we don’t really much care about the truth of what we’re saying, and scientists have to be extremely self conscious about this: not just to be interesting but to be right. Most humans most of the time though adopt ideologies and beliefs that are there principally to make their minds attractive to others, not because those beliefs actually correspond to the world.

John P. A. Ioannidis provides the proof of widespread research bias in
<a href="http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1182327"
Why Most Published Research Findings Are False.

Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.

The factors identified by Ioannidis contributing to bias include:

  • studies conducted in a field are smaller;
  • when effect sizes are smaller;
  • when there is a greater number and lesser preselection of tested relationships;
  • where there is greater flexibility in designs, definitions, outcomes, and analytical modes;
  • when there is greater financial and other interest and prejudice;
  • and when more teams are involved in a scientific field in chase of statistical significance.

All of these factors apply to global warming and global warming effects science: the small effect, the significance chasing, the ad hoc methodologies, the competition, and of course the financial and other interests. All adding up to the increased probability of Type 1 error, of accepting a difference with none actually exists. So the strategy of skeptics is invariably constrained to saying ‘hold on, you have inflated certainty here, or made this mistake there’. But of the factors above, the only one that is really amenable to change that could counteract human bias is the fourth, greater standardization in designs and analytical modes.

This is where replication, checking, data access, and the service of the Accredited Statistician comes to the fore. This view is promoted in the recent article by McCullough and McKitrick entitled: Check the Numbers: The Case for Due Diligence in Policy Formation. This is something Steve McIntyre and myself, Ian Castles and others have been harping on for years, and the value should be a slam-dunk in the current spate of investment frauds. Its all about the numbers. Notably, their 44 page report has a section on Droughts in Australia (pp27).

In July 2008, the Australian Bureau of Meteorology and the Commonwealth Science and Industrial Research Organization (CSIRO) released a report entitled An Assessment of the Impact of Climate Change on the Nature and Frequency of Exceptional Climatic Events. It received considerable media attention for what appeared to be predictions of a dramatic increase in drought. News coverage by the Australian Broadcasting Corporation began, “A new report is predicting a dramatic loss of soil moisture, increased evaporation and reduced ground water levels across much of Australia’s farming regions, as temperatures begin to rise exponentially” (ABC Rural, July 7, 2008).

Shortly after its release, David Stockwell, an ecological systems modeler and Australian expatriate living in San Diego, became doubtful about whether the models had any demonstrated ability to predict known past events and whether the forecast changes were statistically significant–i.e., distinguishable from random guesses. However, neither the data nor the methodology were sufficiently well described in the report to allow him to investigate. Stockwell emailed CSIRO to request the data used for the claims in the report. The request was promptly refused. He was told on July 15, 2008, that the data would not be sent to him “due to restrictions on Intellectual Property” (Niche Modeling, July 15, 2008). About a month after Stockwell’s requests began to get media and Internet attention, CSIRO changed course and released their data. Stockwell quickly found that the models were unable to replicate observed historical trends, typically generating patterns that were opposite to those in the data. Hence their predictions of future trends did not have the credibility CSIRO had claimed (Niche Modeling, August 28th, 2008). By this time, however, media interest in the report had substantially died away so the initial impression was not corrected.

David Karoly

While Prof. David Karoly’s guest post at RealClimate is admirably nuanced, he has graciously left some low-hanging fruit to take the stick to. He states:

1. Increases of mean temperature and mean maximum temperature in Australia have been attributed to anthropogenic climate change, as reported in the IPCC Fourth Assessment.

In a comment at ClimateAudit, Ian Castles remarked on a a similar temperature phenomenon attributed to David Karoly, but mysteriously dissapearing from the AR4:

As long ago as 1996 Professor David Karoly (subsequently a lead author of the TAR and AR4) included “reduction in the diurnal temperature range” (DTR) among the pieces of evidence that had led the IPCC to reach the conclusion that “the balance of evidence suggests a human influence on climate” (“Detecting a Human Influence on Climate” in Australian National Academies? Forum conference “Australians and Our Changing Climate: Past Experience and Future Destiny”, 25 November 1996, Summary of Proceedings, p. 39).

Now that it is recognized that the DTR did not change between 1979 and 2004, these confident assessments can no longer be sustained: there is no reference to the phenomenon in Table SPM-2 (p. in the SPM of the AR4. If models show a faster increase in nighttime temperatures in the last decades of the twentieth century, as stated in Chapter 8, the authors? confidence in the models should have been weakened (whereas the main theme of the Chapter is that confidence in the models has increased since the TAR).

While anybody can be wrong, or change their minds, it is interesting to see how inconvenient mistakes are swept under the rug.

2. While south-east Australia is expected to have reduced rainfall and more droughts due to anthropogenic climate change,

3. In addition, reduced rainfall and low relative humidity are expected in
southern Australia due to anthropogenic climate change.

In what was an otherwise immaculately referenced article, this assertion was strangely not referenced. It makes the dubious claim that due to a globally widespread phenomenon, a tiny localized area is expected (by whom?) to have lower rainfall. I wonder where it came from?

4. it is clear that climate change is increasing the likelihood of environmental conditions associated with extreme fire danger

Global warming is largely equivalent to the poleward movement of climate zones, and consequently vegetation types adapted to those zones. To anticipate the future fire danger in Victoria, one has only to look at the fire danger to the immediate north. I don’t know Victoria, but I bet the vegetation types to the north have a lower fire risk than the montane forests of Gippsland. At equilibrium the vegetation types will adjust and fire risk will reduce to those typical of a warmer, dryer climate composed of lower, more open forest. Temporary disequilibrium in the form of tall closed forests in a warm dry climate is probably the greatest contributor to high fire risk. Over time, as equilibrium reestablishes, the risk will return to normal.

Drought Exceptional Circumstances

Ian Castles organized a review of the Drought Exceptional Circumstances Report by two Accredited Statisticians, who also review my first report on the skill of the climate models.

The statisticians find inadequate validation of the models of drought, as well as suboptimal regionalization in the DECR. They also find my analysis lacked force, and so I have done additional analysis in line with their suggestions.

The last few posts in the series have consisted of reviews of an unsuccessful submission to the Australian Meteorological Magazine (AMM), showing how contradictions between models and observations were suppressed from the conclusions of the DECR. These reviews cover similar ground from a different angle: the skill of the climate models in the DECR, failing to identify any real skill in the predictions of drought, and ways of showing variation between the model (increasing drought) and their real world observations (decreasing drought) at the climatic time scale.

Below are the abstracts:

Some comments on the Drought Exceptional Circumstances Report (DECR) and on Dr David Stockwell’s critique of it

K.R.W. Brewer1 and A.N. Other1
28 January, 2009

1. K.R.W. Brewer is an Accredited Statistician of the Statistical Society of Australia Inc. (SSAI) and a long term Visiting Fellow at the School of Finance and Applied Statistics within the College of Business and Economics at the Australian National University.
2. A.N. Other is a pseudonym for another Accredited Statistician of the SSAI who prefers to remain anonymous. Full responsibility for the content is taken by K.R.W. Brewer.

Abstract

The Drought Exceptional Circumstances Report (DECR) was authored by a team drawn from the CSIRO and Australia’s Bureau of Meteorology, and was publicly released in July 2008. Almost immediately it became a source of controversy. This evaluation, both of the Report itself and of the critique of it written by Dr David Stockwell, finds good mixed with less than good in both. The DECR itself is criticized for its poor delineation of Regions within Australia, for the choices made of statistics to be constructed, for the manners of their construction, and for not getting the best out of the relevant available data. Dr Stockwell is criticized for his inappropriate choices of methodology and of time periods for analysis, and also for misunderstanding some parts of what the DECR’s authors had chosen to do. Nevertheless, both the Report itself and Dr Stockwell’s critique of it are welcome stimuli to further investigate a serious issue within the climate change debate.

Validation of Climate Effect Models: Response to Brewer and Other

David R.B. Stockwell
February 4, 2009

Abstract

A review by independent Accredited Statisticians, Brewer and Other [KB09], suggested that some claims in the report “Tests of Regional Climate Model Validity in the Drought Exceptional Circumstances Report” [DS08] were premature. Additional tests suggested by KB09 support the claim made in the original report of “no credible basis for the claims of increasing frequency of Exceptional Circumstances declarations”. The contributions of KB09 and DS08 to the evaluation of skill of climate model simulations with, arguably, weakly validated idiosyncratic statistics are discussed. These include recommendations for greater rigor in evaluating the performance of climate effects simulations, such as those used in standardized forecasting practices [AG09].

One thing is clear, the climate models that all of these predictions rely on have not been validated to accepted standards. That is a major lapse on the part of the climatologists who nonetheless use the models to influence public opinion and action.

Contrast the quality and professionalism of the review by statisticians, with the error-ridden categorical reviews by climate scientists to the AMM article. The greater rigor of the statisticians is clearly evident.

Continue reading Drought Exceptional Circumstances

Validation of Climate Effect Models: Response to Brewer and Other

David R.B. Stockwell
February 4, 2009

Abstract

A review by independent Accredited Statisticians, Brewer and Other [KB09], suggested that some claims in the report “Tests of Regional Climate Model Validity in the Drought Exceptional Circumstances Report” [DS08] were premature. Additional tests suggested by KB09 support the claim made in the original report of “no credible basis for the claims of increasing frequency of Exceptional Circumstances declarations”. The contributions of KB09 and DS08 to the evaluation of skill of climate model simulations with, arguably, weakly validated idiosyncratic statistics are discussed. These include recommendations for more rigor in evaluating the performance of climate effects simulations, such as those used in standardized forecasting practices [AG09].

Introduction

As part of a review of the support to farmers and rural communities made under the Exceptional Circumstances (EC) arrangements and other drought programs, the Australian Federal Government Department of Agriculture, Fisheries and Forestry (DAFF), commissioned a study from the CSIRO Climate Adaptation Flagship and Australian Bureau of Meteorology (BoM) to examine the future of EC declarations under climate change scenarios. The DECR report examined yearly percentage area affected by exceptional temperature, rainfall, and soil moisture levels for each of seven Australian regions from 1900 to 2007 for both recorded observation and climate models (projecting, simulating or forecasting) historic drought, concluding:

DECR: Under the high scenario, EC declarations would likely be triggered about twice as often and over twice the area in all regions.

The interpretation of such statements by their client, the Department of Agriculture, Fisheries and Forestry (DAFF), is illustrated by a press release 6 July 2008 (DAFF08/084B) stating:

DAFF: Australia could experience drought twice as often and the events will be twice as severe within 20 to 30 years, according to a new Bureau of Meteorology and CSIRO report.

After the summary data used in the DECR report was made freely available on the BoM website, an assessment of the validity of the climate models was circulated [DS08] examining the skill of climate models, concluding there was “no basis for belief in the claim of increasing frequency of EC declarations”. At the initiation of Dr Ian Castles, independent accredited statisticians reviewed the DECR and DS08 [KB09]. This study provides some additional analysis in response to suggestions in KB09, and addresses other questions regarding DS08. The analysis is available in an R script [R09].

Why Validate?

A model or simulation like a global climate model (GCM) is a surrogate for an actual climate system. If the model does not provide a valid representation of the actual system, any conclusions derived from the model or simulation are likely to be erroneous and may result in poor decisions being made. Validation of models is expected practice throughout all society, similar to be business concept of ‘fitness for use’. Specifically:

Validation is the process of determining the degree to which a model or simulation is an accurate representation of the real world from the perspective of the intended uses of the model or simulation.

Validation consists of comparing simulation and system output data over one or more statistics with a formal statistical procedure. Examples of statistics that might be used include the mean, the trend, and correlation or confidence intervals. It is important to specify which statistics are most important, as some statistics may be more relevant than others. In the case of forecasting effects of CO2 on drought, the overall trend is regarded as more important than the patterns of correlation, because climate is a longer term phenomenon.

A model’s results have credibility if they satisfy additional factors: demonstration that the simulation has been validated and verified, and general understanding and agreement with the simulation’s assumption. If validity has not been demonstrated adequately, or the model ‘fails’ in key ways, then it is not ‘fit for use’. If it fails all tests, then it is accurately described as ‘useless’, and certainly cannot be regarded as credible.

Discussion

KB09 agrees with DS08 on need for more effective validation of models of droughts at regional scales.

6. Dr Stockwell has argued that the GCMs should be subject to testing of their adequacy using historical or external data. We agree that this should be undertaken as a matter of course by all modellers. It is not clear from the DECR whether or not any such validation analyses have been undertaken by CSIRO/BoM. If they have, we urge CSIRO/BoM make the results available so that readers can make their own judgments as to the accuracy of the forecasts. If they have not, we urge them to undertake some.

7. If any such re-evaluation is to be carried out, however, it should be done using two separate time periods, namely 1900-1950 (during which the rainfall trend was generally upwards) and 1950-2007 (where it was generally downwards.) This would allow the earlier period to provide internal validations and the later period external validations. However, if and when these analyses are repeated, the raw data used should be compiled not for the existing seven Regions, but for more homogeneous Regions, as suggested in item 1 above.

Note that the thrust of DS08 was that models failed a range of validation tests, so there was no credible basis for the DECR claims. Additional analyses follow using the time periods suggested in KB09, and reporting the normality of distributions and residuals. These analyses use robust tests on mean values of each year over all regions and models in order to improve the normality of the distribution by filling in most of the zero (no drought) years. While aware there remain deficiencies in the approach: eg. the mean is over regions of unequal size, as we were not supplied with grid cells for making true means, it is argued the result is robust.

drought-obs

Fig 1. The area of each of the seven regions under exceptionally low rainfall (colors), and the mean (black).

Difference of means 1900-1950 vs. 1950-2007

Table 1 is similar to the mean comparison analysis in RBHS06. Here, for observations, the mean of droughted area in all 7 regions over the period, and for model projections, the means of all 7 regions calculated from the means of all 13 models area were compared over half-century periods. The mean areal extent of observed exceptionally low rainfall years decreased from 1900-1950 to 1951-2007, while the simulated area of exceptionally low rainfall years increased over the same period. The p values for a non-parametric Mann-Whitney test, used because the observations are not normally distributed, indicate the differences between the periods are highly significant.

Table 1: Mean percentage area of exceptionally low rainfall over time periods suggested by KB09. A Mann Whitney rank-sum test shows significant differences between periods.

1900-2007 1900-1967 1951-2007 P 1900-2007 vs. 1951-2007 P 1900-1950 vs. 1951-2007 Test
Observed % Area Drought 5.6±0.5 6.2±0.7 4.9±0.6 0.10 0.004 Mann-Whitney test
(wilcox.test(x,y) in R)
Modelled % Area Drought 5.5±0.1 4.8±0.2 6.2±0.2 0.006 <0.001 Mann-Whitney test
(wilcox.test(x,y) in R)

Trends in 1900-1950 vs. 1950-2007

Table 2 below shows two analyses related to trends on the entire data set, and the p-value from a Shapiro test for normality of residuals. A significant negative coefficient in LM Obs vs Exp 1900-2007 indicates an inverse relationship between observations and forecasts, while the significant p-value in the Shapiro test indicates residuals are not normally distributed. While the trend of the observations of drought area over the 1951-2007 period is not significantly different to zero in this test, the trend of the projections over the same period is positive and significant. The Shapiro tests are significant, indicating non-normality of residuals.

These results are consistent with those obtained by a different method in Table 1. The models forecast increasing drought areas, but the trend in observations of drought extent are mildly or significantly decreasing. Taking the mean of all models and regions, did not correct departure of residuals from normal due to the highly non-normal original data distribution. Normal residuals may only be obtained with a greatly improved statistical modelling approach, beyond the scope of this study.

Table 2. Linear regression test and residual normality of (1) all observed and forecast data and (2) trends of mean of observed and forecasts:

Linear model Shapiro test
LM Obs vs. Exp 1900-2007 Obs = -0.6*Exp + 8.9
r2=0.04 p=0.04
P<0.001
LM of Obs 1951-2007 Obs = -0.02*Exp + 6.3
R2 = -0.01 p=0.78
P<0.001
LM Forecast 1951-2007 Obs = 0.04*Exp + 2.7
R2 = 0.07 p=0.06
P<0.001

Moving 30 year Averages

Another approach to evaluating climatic trends was illustrated in the DECR and in Fig 1 from DS08. Fig 2 below shows the overall 30-year running mean of percentage area of exceptionally low rainfall for observations decreasing in almost all areas, and forecasts increasing in all areas. Further visual evidence of the significance of the difference between model projections and observations is shown by the lack of overlap of the spread of results at 1990.

image003

Figure 2. Overall average (green thick line) of the 30-year running average of percentage area of exceptionally low rainfall for observations is decreasing, in almost all areas (red lines), while models (black lines) are increasing in all areas.

No doubt other statistics could be used to compare the difference of observed and modelled drought trends, with greater confidence if normality could be achieved. The most accurate conclusion then is that while it may be premature to say the models are entirely ‘useless’ at simulating drought, they have not been shown to be ‘fit for use’ of forecasting trends, and so are not credible.

Weather, Climate and Chaos

KB09 suggested there were misunderstandings of the DECR in the DS08 review (without being specific as the word is not used elsewhere in the report). Possibly KB08 refer to a distinction between ‘weather’ vs. ‘climate’ in the sense used by Andrew Ash below (pers. comm.).

AA: The correlation and efficiency tests are based on an assumption that the climate models are being used to hindcast historical weather. This assumption is incorrect.

Their argument is that the failure of validation at shorter time scales of weather does not block the fitness of the model at a 30 year time scale. I was because of this distinction, more emphasis was placed on the trends in DS08 (see Fig 2) as shown by the statement in the abstract of DS08:

DS: The most worrying failure was that simulations showed increases in droughted area over the last century in all regions, while the observed trends in drought decreased in five of the seven regions identified in the CSIRO/Bureau of Meteorology report.

Further, Fig 10 performs a crude validation by showing the variability of low rainfall lies within the range simulated by the multi-model ensemble (MME) of the models. The rationale is that because the observed temperature and rainfall are random instantiations of highly chaotic trajectories, the observations are comparable only with a specific model simulations. There are a number of problems with this view:

  1. The selection of the 13 climate models is ad hoc; and hence no assurance the MME properly samples the relevant state space. As a result, MMEs are sometimes referred to pejoratively as “ensembles of opportunity” [PD08].
  2. Even if the MME can be regarded as ‘skillful’ by virtue of containing the observations, this test does not demonstrate skill of models at forecasting trends. For that, one would need to demonstrate the models can match the trends in the observations.
  3. If validation only requires that the observations stay within the full range of all individual model simulations, where the models are of unknown accuracy, then skillful models are indistinguishable unskilful ones.
  4. As the correlation of trends in CO2 and temperature over the last 50 years is widely regarded as evidence of warming due to CO2, it is inconsistent to claim that a difference in the trend of warming and drought over the same time scale is inconsequential.
  5. Fig 2 is suggestive that the ranges predicted by the models, and the range of drought frequency may have in fact diverged significantly.

Some of these issues are highly technical but require closer evaluation to see what is actually being validated in an MME. If observations such as rainfall must only lie within the range simulated my the models, all that is being tested is the ability of the models to simulate the range (or variance) of the observations. Therefore, one cannot presume such models can also successfully simulate other features, such the mean value of the observations, the change in the mean value, the trend of the observations and so on.

It is crucial that if the intended ‘fitness for use’ of a model is to forecast trends, then validation must consist of demonstrated skill at modelling trends in historic data. This is the conventional view, and the view expressed in DS08, KB09, that evidence is necessary for supporting claims. One should also remark on the wisdom of the old saw, that extraordinary claims require extraordinary evidence. The DECR made an extraordinary claim about the change in the trend of the observations:

EC declarations would likely be triggered about twice as often and over twice the area in all regions

No evidence has been demonstrated of even ordinary evidence of skill at their intended use. Another way to say this, is that the validation consisting of the MME enclosing the range of observations is a very weak test, so weak that very little can be reliably inferred from it.

Tests of Individual Models

KB09 are concerned with the force of arguments in DS08, especially the use of the word ‘significant’ where residuals may not have been normal. In retrospect I would have qualified the word more. It is not clear the extent to which lack of normality of residuals undermines results, and departure from normality is quite common. Unfortunately, normality of residuals may be difficult to achieve with this type of data, without using much more sophisticated approaches.

KB09 outlined a preferred approach but performed no analysis:

9. A Possible Alternative to OLS Regression. It is at least possible that forecasting using simple ARIMA modelling [3], [4], might prove to be just as accurate and far easier to justify than OLS regression.

The DECR rainfall analysis uses a peculiar metric: the percentage of area with rainfall below the 5th percentile. In formal terms, this appears to be a ’bounded extreme value, peaks over threshold’ statistic. The distribution resembles a Pareto (power) law, but due to the boundedness where predicted extent of drought approaches 100%, the distribution becomes more like a beta functions.

KB09 believe a statistic such as average annual rainfall may preserve more information. In this case, the residuals of standard tests in DS08 might also improve. By way of explanation for the use of standard tests, there are ‘hard yards’ in developing a formal statistical model such as KB09 propose, on such an idiosyncratic statistic. The pragmatic approach of DS08 was to use a number of statistics breaking down the relevant elements into questions as follows:

  1. How much of the variation in the observations is explained by models? – R2 Correlation
  2. Does trend in drought severity and frequency match models? – slope of linear regression
  3. Do the models agree with the historic severity of droughts? – Nash-Sutcliffe coefficient.
  4. Do the models agree with the historic frequency of droughts? – return period.

Below are more specific notes on each of their concerns, largely framed in the form of if-then-maybe hypothetical:

If certain changes were made to the DECR analysis, then maybe the results might not have been so bad.

DS08 assessed the DECR as received, while KB09 is more constructive and speculative. This raises a larger question of how we go about assessing an idiosyncratic model, a topic expanded on in the conclusions.

Trends

Claims of significance of idiosyncratic data with non-normal distributions and autocorrelation do need to be treated with caution. KB09 take umbrage with a linear fit to the whole time period from 1900 to 2007, and argue that if a shorter time frame such as the period 1950-2007 were examined, then maybe results might improve. One might also argue this approach is arbitrary and informed by prior examination, or ‘data snooping’.
In defense of using the full period, CO2 and temperatures are both generally increasing from 1900 to 2007. Hence, any CO2 or temperature related correlations with drought should be discovered. Nevertheless, Fig 1 and Tables 1 and 2 shows identical results that the inverse relationship between trends in the models and observations exists at different time intervals.
Regarding the comment in KB09 that p values seem too low, while standard deviations (s.d.) were quoted in Table 1, the standard error of the mean (s.e.) was used to calculate the p values, as stated in the caption “Table 1: t-test of difference in mean of predicted trends to the observed mean of droughted area”. This follows the practise of DCPS07 and interested readers can refer to DD07 and associated links for discussion.

R2

Consistently small values in the r2 columns of Tables 2-8, indicate lack of variance explained by the models. Here KB09 state:

If the implication is that the GCM-based projections do not reflect year to year changes in the drought affected percentages of the seven Regions, we do not regard this as a serious failure. It is not what the GCMs were constructed to do. They were meant to indicate long term trends.

This statement seems informed by a view prevalent in climate science that the r2 statistic only explains year-to-year variance and hence is invalid at climatic scales. However, this view is misleading. The r2 statistics will robustly quantify variation at a range of scales, including short and long term trends. R2 was used more in this capacity as a robust detector of possible skill. As we see only 1% of all variation (both short and long term) is explained by the models.

Frequency

The average time between droughts in each region differed between observations and models. KB09 suggest that a “fallacy of composition” effect makes it more than possible that the two calculations of return period could be widely different for reasons other than lack of skill of models. This was confirmed by Andrew Ash:

AA: The observed data have the shortest return period as they have the finest spatial resolution and the model based return regions have increasingly larger mean return periods, inversely related to the spatial resolution at which they are reported.

Nevertheless, the labels on the data sets indicate the supplied data for models and observations represent comparable quantities at the same comparable, regional, scale: percentage area below the 5th percentile of exceptionally low rainfall. To compare on the same grid cell basis, we would have needed access to both observed and projected rainfall data within 25km grid cells, which we were not supplied with. Even so, this could be regarded as an “if then maybe” objection, if the analysis were conducted on the same scale then maybe skill would be shown in drought frequency. Then again, maybe not.

Severity

Consistently negative values of the Nash-Sutcliffe coefficient in Table 2-8 imply that “If averaged over time, each of the 13 GCMs’ sets of projections lies further away from the corresponding set of observed values than the simple mean of the observed values do.” KB09 suggest that if analysis were conducted for another period where the net change in rainfall was not constant, then perhaps the result would not be so bad.

The Nash-Sutcliffe coefficient is widely used in hydrological models for assessing the quality of model outputs against observations. As far as I know, it should not be affected by the start and end points of the series, rather performing a kind of sum of squares on the difference between observed and projected values at each point. Otherwise, my comments on choice of period of analysis also apply.

Conclusion

KB09’s main concerns may be summarized as: (1) some tests do not appear consistent with assumptions, and (2) DS08 did not eliminate all possible explanations for poor results, attributing poor results entirely to lack of model skill. New tests suggested by KB09 show the strong and significant departure of model projections from the observed pattern of historic droughts, with a strong bias in favor of increased and increasing drought in Australia with increasing levels of CO2. These additional analyses agree with the findings in DS08, demonstrating the robustness of the findings. Thus it appears the claim of no credible basis for increasing droughts, is not affected and actually vindicated by KB09’s report.
In DS08 the KB09 recommendation regarding improved statistical models and regionalization were alluded to in the discussion 2:

DS: Recasting the drought modelling problem into known statistical methods might salvage some data from the DEC report. Aggregating the percentage area under drought to the whole of Australia might reduce the boundedness of the distribution, and might also improve the efficiency of the models.

While drought biased climate simulations play well during a severe drought in the political power-bases of the country, the practice of uncritical acceptance of unvalidated or invalid must be strongly discouraged in evidence-based science policy. Finally, to quote Luboš Motl [LM08]:

And perhaps, most people will prefer to say “I don’t know” about questions that they can’t answer, instead of emitting “courageous” but random and rationally unsubstantiated guesses.

Further Work

One avenue for further work is the development of an ARIMA or other statistical framework for areas of exceptionally low rainfall as suggested by KB09.

Preliminary split sample analysis described at NM08 could be developed. These results suggest GCM models cannot be ‘selected’ on the basis of their historic fit to drought at regional scales. In most areas the models that do well in one 50 year period do poorly in another, and vice versa, further indicating ‘failure’ in external validation. The low value of GCM’s for regional effects forecasting are not fully discounted by their promoters.

Another avenue of inquiry is robust statistics for assessing the confidence in idiosyncratic models, where the developers performed no detailed statistical modelling. As such studies have no assumptions that conform to more standard approaches, most standard tests are going to be formally invalid. These concerns would argue for more agreed upon metrics of performance such as those proposed for forecasting [AG09].

References

[AG09] “Analysis of the U.S. Environmental Protection Agency’s Advanced Notice of Proposed Rulemaking for Greenhouse Gases”, Drs. J. Scott Armstrong and Kesten C. Green a statement prepared for US Senator Inhofe for an analysis of the US EPA’s proposed policies for greenhouse gases. http://theclimatebet.com

[CL05] Cohn, T. A., and H. F. Lins (2005), Nature’s style: Naturally trendy, Geophys. Res. Lett., 32(23), L23402, doi:10.1029/2005GL024476.

[DCPS07] David H. Douglass, John R. Christy, Benjamin D. Pearsona and S. Fred Singerc,det al. (2007). “A comparison of tropical temperature trends with model predictions” (PDF). International Journal of Climatology 9999 (9999): 1693. doi:10.1002/joc.1651. http://icecap.us/images/uploads/DOUGLASPAPER.pdf. Retrieved on 12 May 2008.

[DD07] David Douglass’ Comments: http://www.climateaudit.org/?p=3058

[DECR] Drought Exceptional Circumstances Report (2008), Hennessy K., R. Fawcett, D. Kirono, F. Mpelasoka, D. Jones, J. Batholsa, P. Whetton, M. Stafford Smith, M. Howden, C. Mitchell, and N. Plummer. 2008. An assessment of the impact of climate change on the nature and frequency of exceptional climatic events. Technical report, CSIRO and the Australian Bureau of Meteorology for the Australian Bureau of Rural Sciences, 33pp. http://www.daff.gov.au/__data/assets/pdf_file/0007/721285/csiro-bom-report-future-droughts.pdf

[DS08] David R.B. Stockwell, Tests of Regional Climate Model Validity in the Drought Exceptional Circumstances Report – landshape.org/stats/wp-content/uploads/2008/08/article.pdf

[KB09] K.R.W. Brewer and A.N. Other, (2009) Some comments on the Drought Exceptional Circumstances Report (DECR) and on Dr David Stockwell’s critique of it.

[KM07] Koutsoyiannis, D., and A. Montanari, Statistical analysis of hydroclimatic time series: Uncertainty and insights, Water Resources Research, 43 (5), W05429.1–9, 2007.

[LM08] Lubos Motl, The Reference Frame – http://motls.blogspot.com/

[NM08] Niche Modelling – http://landshape.org/enm/temperature-index-drought/

[PD08] T.N. Palmer, F.J. Doblas-Reyes, A. Weisheimer, G.J. Shutts, J. Berner, J.M. Murphy, Towards the Probabilistic Earth-System Model, arXiv:0812.1074v2 [physics.ao-ph]

[R09] R script for analysis

[RBHS06] Rybski, D., A. Bunde, S. Havlin, and H. von Storch (2006), Long-term persistence in climate and the detection problem, Geophys. Res. Lett., 33, L06718, doi:10.1029/2005GL025591.

Some comments on the Drought Exceptional Circumstances Report (DECR) and on Dr David Stockwell’s critique of it

K.R.W. Brewer1 and A.N. Other1
28 January, 2009

1. K.R.W. Brewer is an Accredited Statistician of the Statistical Society of Australia Inc. (SSAI) and a long term Visiting Fellow at the School of Finance and Applied Statistics within the College of Business and Economics at the Australian National University.
2. A.N. Other is a pseudonym for another Accredited Statistician of the SSAI who prefers to remain anonymous. Full responsibility for the content is taken by K.R.W. Brewer.

Abstract

The Drought Exceptional Circumstances Report (DECR) was authored by a team drawn from the CSIRO and Australia’s Bureau of Meteorology, and was publicly released in July 2008. Almost immediately it became a source of controversy. This evaluation, both of the Report itself and of the critique of it written by Dr David Stockwell, finds good mixed with less than good in both. The DECR itself is criticized for its poor delineation of Regions within Australia, for the choices made of statistics to be constructed, for the manners of their construction, and for not getting the best out of the relevant available data. Dr Stockwell is criticized for his inappropriate choices of methodology and of time periods for analysis, and also for misunderstanding some parts of what the DECR’s authors had chosen to do. Nevertheless, both the Report itself and Dr Stockwell’s critique of it are welcome stimuli to further investigate a serious issue within the climate change debate.

Continue reading Some comments on the Drought Exceptional Circumstances Report (DECR) and on Dr David Stockwell’s critique of it

Just Not Cricket

Here we go again. FBI sweeps into cricket boss Sir Allen Stanford’s bank.

Agents are examining the Antigua-based Stanford International Bank (SIB), which has paid investors returns twice as large as conventional banks, after former employees said they witnessed “unethical and illegal practices”.

Learn to look for the numbers. When you see unusually high returns, you need to disbelieve them immediately. When you see extrordinary claims in science, such as high values of significance, or any predictions of large deviations from the norm, immediately become very skeptical.

Flamboyance and controversy are Sir Allen Stanford’s hallmarks. He announced his inaugural Twenty20 cricket tournament by landing at Lord’s in a gold-plated helicopter. After concern from Buckingham Palace, he recently changed a page on his website that said he was knighted by the Earl of Wessex. The honour was in fact bestowed by Antigua’s government.

Also see Climate Predictions are Unpredictable for a roundup of the backlash over climate science fool’s gold.

Letters

Below are Peter Gallagher’s thoughts on the reviews of the submission to AMM. Contrast this with ac’s impressions that “To my reading the reviewer’s criticisms are reasonable and pertinent.” It goes to show, that reasonable and unrelated people can see things in different ways. Where is the resolvability of fact in the review process? Consensus?

Hi David,

Thanks for sending me these papers.

Reading the reviews, it seems to me that your submission has been poorly understood by the reviewers (or not properly characterized by the editor). The reviewers have treated it as though it were an academic journal review rather than as a ‘note’ or ‘commentary’.

Also, consciously or not, they have ignored a crucial dimension of the DECR report: its role in the policy dialog (or ‘fit up’) on Exceptional Circumstances drought relief.

This is a categorical mistake: like reading the Christian gospels as a biography.

In the full context of the DECR, the narrowness of your focus is entirely justified and the nature of your references and footnotes irrelevant. What matters is the accuracy of your criticisms of the conclusions that have been picked up and magnified by the Prime Minister.

Your paper seems to me to demonstrate your points:

(a) The models on which the DECR relies have no ‘skill’ in reproducing rainfall records and therefore are completely inadequate as the basis for projections.

(b) The melding of observations and projections in the DECR report has been obscured ( a ‘Harry-Gill’ error?)

(c) The reliance only on the 10th percentile scenario (and it’s re-labeling as the ‘high’ scenario) in the summary seems actually misleading in view of the mean results across all scenarios.

(I’m amused by the self-satisfaction in one review: We don’t need external reviews because our own are top notch in our view).

Kind regards,

Peter Gallagher

Audits and transparency

Mish reports on a massive secret audit of the major US banks. He raises some interesting questions, relevant to auditing and secrecy, and both in finance and in science.

Nearly 100 federal banking regulators descended on Citigroup in New York on Wednesday morning. Dozens more fanned out through Bank of America, JPMorgan Chase and other big banks across the nation.

There is no transparency and there are no details. Is this supposed to inspire confidence?

How can there possibly be any trust in a system that may or may not be taken over by the government, under unspecified conditions, with unspecified strings, when there is no transparency as to what is happening anywhere along the line?

I have no problem with the audit. I have a massive problem with Geithner’s intention to hide the results and I have a massive problem with squandering taxpayer money to bailout banks whose greed was a big part of the problem. Shareholders and bondholders of failed institutions should be wiped out and management of failed banks should be fired and replaced with executives of banks that avoided these problems.

Examples of Research Bias

The Financial Times recently reported on the Australian bushfires, linking them to increases in greenhouse gases. We take another look at the data in the DECR and find Australia is getting wetter not drier:

Scientists say Australia, with its harsh environment, is set to be one of the nations most affected by climate change.

“Continued increases in greenhouse gases will lead to further warming and drier conditions in southern Australia, so the [fire] risks are likely to slightly worsen,” Kevin Hennessy at the Commonwealth Scientific and Industrial Research Centre told Reuters.

Bob Brown, the senator who leads the Greens party, said the bushfires provided stark evidence of what climate change could mean.

“Global warming is predicted to make this sort of event happen 25 per cent, 50 per cent more,” he said. “It’s a sobering reminder of the need for this nation and the whole world to act and put at a priority our need to tackle climate change.”

The Drought Exceptional Circumstances Report, that I have been reviewing in this series, promoted these conclusions. Lets look at another analysis, this time using simple quantile analysis of the data in Table 3. This table contains the average percentage area having exceptionally low rainfall years for selected 40-year periods and the most recent
decade (1998-2007).


1900-1939 1910-1949 1920-1959 1930-1969 1940-1979 1950-1989 1960-1999 1968-2007 1998-2007
Qld 9.5 6.5 5.5 4.1 3.3 3.1 2.7 2.6 4.7
NSW 5.7 6.9 5.7 6.2 5.8 4.3 4.0 3.8 6.4
Vic&Tas 5.3 6.0 4.2 6.1 5.1 5.0 5.3 5.2 8.5
SW 5.2 7.1 7.2 6.9 7.9 5.9 4.9 4.4 3.4
NW* 6.3 5.3 6.5 7.5 6.5 6.1 4.7 3.5 3.3
MDB 6.1 7.2 5.8 6.4 5.7 4.1 3.5 3.5 6.9
SWWA 2.5 4.7 4.1 6.5 8.3 6.1 6.3 8.5 8.9
Australia 6.4 6.4 6.6 6.4 6.3 5.3 4.6 3.5 3.1

Using the function ‘quantile’ in R, we output the percentage areas for each probability in each 40 year period. Then we lookup the probability for each region using the most recent 40 year period 1968-2007.


Quantiles
5% 10% 50% 90% 95%
3.25 4.05 5.85 7.15 7.60


Regions, area and probability drought has increased.
Qld 2.6 <5%
NSW 3.8 <10%
Vic&Tas 5.2 NS
SW 4.4 NS
NW* 3.5 <10%
MDB 3.5 95%
Australia 3.5 <10%

The results show that over the last 40 years, regions Qld, NSW, NW, and MDB have had significantly less area under drought. Only in SWWA has the drought area increased significantly, while Vic&Tas (the region of recent bushfires) and SW have no significant change.

The ‘inconvenient’ results were reported in the DECR text as follows:

Observed trends in exceptionally low rainfall years are highly dependent on the period of analysis due to large variability between decades.

Despite these highly significant DECR results showing Australia getting wetter, not drier, CSIRO scientists continue to report in the media that Australia will get drier.

It only takes two thoughts to realize that wetter conditions can pose greater fire risks due to the greater production of fuel in the wet season, and more dangerous conditions when it drys out. Drier conditions lead to a more open grassland environment in Australia, much like the African Savannah, with cooler grassfires but not the hot forest fires suffered recently in Victoria. You simply cannot look at environmental factors in isolation.

But don’t tell CSIRO, or the next thing we will hear is that greenhouse gases are causing more fires by making it wetter.

Do increases in greenhouse gases cause droughts in Australia?

Peter Gallagher reports that even while the coals are still warm, some are already blaming the Victorian fires on increases in greenhouse gases.

The following summarizes indications of decline in droughts in Australia from 1900 to the present, compiled from data provided with the Drought Exceptional Circumstances Report. Some of this information was provided in the submission the the Australian Meteorological Magazine (more about this tomorrow). Drought is defined as the percentage of area with rainfall lower than the 5th percentile. The areas are averaged over seven Australian regions.
Continue reading Do increases in greenhouse gases cause droughts in Australia?

Climate Flagship Response

A number of familiar tests, often used to evaluate the performance of models: R2 correlation, Nash-Sutcliffe efficiency and similarity of trends and return period, were reported here, noting not much evidence of skill in the DECR models compared with observations at any of these. I also said what a better treatment might entail but left that for another time:

The percentage of droughted area appears to be a ’bounded extreme value, peaks over threshold’ or bounded POT statistic. The distribution resembles a Pareto (power) law, but due to the boundedness where predicted extent of drought approaches 100% becomes more like a beta distribution (shown for SW-WA on Fig 2). Recasting the drought modeling problem into known statistical methods might salvage some data from the DEC report. Aggregating the percentage area under drought to the whole of Australia might reduce the boundedness of the distribution, and might also improve the efficiency of the models.

Aware that the tests I applied were not the last word due to the idiosyncratic nature of the data, the conclusion in the summary was slightly nuanced: that as there was no demonstration of skill at modeling drought (in both the DECR and my tests), and as validation of models is necessary for credibility, there is no credible basis:

Therefore there is no credible basis for the claims of increasing frequency of Exceptional Circumstances declarations made in the report.

What is needed to provide credibility is demonstrated evidence of model skill.

Andrew Ash, of the CSIRO Climate Flagship sent a response on 12/18/08. This was to fulfill an obligation he made on 16 Sep 2008, to provide a formal response to your review of the Drought Exceptional Circumstances report (dated 3 Sep 2008), after my many requests to provide details of the validation to skill at modelling of droughts.

I must say I was very pleased to see there was no confidentially text at the end of the email. I feel so much more inclined to be friendly without it. I can understand confidentiality inside organizations, but to send out stock riders to outsiders is picayune. The sender should be prepared to stand by what they say in a public forum and not hide behind a legal disclaimer. Good on him for that.

The gist of the whole email is that he felt less compelled to respond due to an ongoing review I sent to the Australian Meteorological Magazine on 23 Sep 2008. As it was, I was still waiting for the review from the AMM on the 18th December when I received this response. Kevin Hennessy relayed some advice from Dr Bill Venables, a prominent CSIRO statistician, and the following didn’t add anything:

However, we have looked at the 3 Sep 2008 version of your review. The four climate model validation tests selected in your analysis are inappropriate and your conclusions are flawed.

* The trend test is invalidly applied because (i) there is a requirement that the trends are linear and (ii) the t-test assumes the residuals are normally distributed. We undertook a more appropriate statistical test. Across 13 models and seven regions, there are no significant differences (at the 5% level) between the observed and simulated trends in exceptionally low rainfall, except for four models in Queensland and one model in NW Australia.

Its well known that different tests can give different results, depending on the test. It is also true that some tests may be better or more reliable than others. Without more details of their more test its hard to say anything, except to say that lack of significance does not demonstrate skill. The variability of the climate model outputs could be so high that they allow ‘anything to be possible’, as often seems to be the case.

* The correlation and efficiency tests are based on an assumption that the climate models are being used to hindcast historical weather. This assumption is incorrect. As a result, the tests selected are inapplicable to the problem being addressed. This in turn leads to false conclusions.

This would be true if these were the only tests and if correlation and efficiency were dependent entirely on ‘short-term-fluctuations’. They are not as they will capture skill at modeling both short AND long term fluctuations. This is also why I placed more emphasis skill on modelling trends over climatically relevant time scales. He is also not specific about which conclusions. The conclusion of ‘no credible basis’ is not falsified by lack of evidence.

It should also be noted that the DECR also considered return periods (Tables 8 and 10) so any criticism of returns periods applies equally to the DECR.

* The return period test is based on your own definition of ‘regional return period’, which is different from the definition used in the DEC report. Nevertheless, your analysis does highlight the importance of data being collected or produced at different resolutions and the effect this has on interpretations of the frequency of drought. The observed data have the shortest return period as they have the finest spatial resolution and the model based return regions have increasingly larger mean return periods, inversely related to the spatial resolution at which they are reported. We were well aware of this issue prior to the commencement of the study and spent a considerable amount of time designing an analysis that would be robust to take this effect into account.

I appreciate the explanation for the lack of skill at modelling return period, which measures drought frequency, as opposed to drought intensity measured by efficiency. Nevertheless, lack of demonstrated skill at modelling drought frequency stands.

Note they continue to be unresponsive to requests for evidence the climate models have skill at modelling droughts. Where we stand at the moment is, that irrespective of the reliability of my tests, there is still no evidence of skill to be seen, at the short term, long term, or at drought intensity or frequency and so my claim of “no credible basis for the claims of increasing frequency of Exceptional Circumstances declarations” still stands. The Climate Flagship has steadfastly abjured presenting validation evidence. While the concerns expressed have relevance to the quality of the tests (which are widely used, but problematic due to the strange data), they were not precise about the conclusions or claims they were trying to rebut.

Further Work

I came across a recent drought study that also finds no statistical significance between models and drought observations. This was actually in Ref 27 of the DECR: Sheffield, J. &Wood, E. F. Projected changes in drought occurrence under future global warming from multi-model, multi-scenario, IPCC AR4 simulations. Climate Dynamics 13, 79-105 (2008).

Although the predicted future changes in drought occurrence are essentially monotonic increasing globally and in many regions, they are generally not statistically different from contemporary climate (as estimated from the 1961-1990 period of the 20C3M simulations) or natural variability (as estimated from the PICNTRL simulations) for multiple decades, in contrast to primary climate variables, such as global mean surface air temperature and precipitation.

Below is a plot of the observations for the drought statistic, area experiencing less than 5% exceptionally low rainfall (leading to an Exceptional Circumstances drought declaration). You can see how ‘peaky’ it is, even when the average is taken (black).

image002

In some ways it might have been better to just knuckle down and develop a POT model right from the start, as it might have allowed me to produce a less nuanced response. I have been doing that, but have had to upgrade R. A recompilation is needed, that took all night and lost my graphic interface to R. Even then, the package VGAM doesn’t compile for some reason, so I have to look for other packages.

DECR Review Series

Posts over the next few weeks will be updates on the status of reviews myself and others have initiated of the Drought Exceptional Circumstances Report (DECR), by the CSIRO and Bureau of Meteorology (BoM).

It is prudent to subject your views to the rigors of peer review. It is the way to knowledge to search out feedback. So I thought why not share the opportunity with others, so can avail themselves of the wisdom of the leading experts, to learn and formulate their own opinion, not only about the DECR itself, the support for increasing drought due to anthropogenic global warming (AGW) from climate models, and the standard of scholarship in climate science.

This series is also a case study in the scientific review process, for examining such issues as the degree to which peers in climate science provide an objective assessment of submissions. I want to stress that, irrespective of differences of opinion, I am deeply grateful for feedback from experts who have spent many years studying the subject matter. It is for that reason, I always analyze the comments of the expert reviewers very carefully for accuracy, as I take on board every worthwhile recommendation.

Below for easy reference are some links to information, data and opinion to date:

The BoM Website listing of the report and selection of intermediate data, used in the analyses.

The Drought Exceptional Circumstances report as downloaded.

DECR: Under the high scenario, EC declarations would likely be triggered about twice as often and over twice the area in all regions.

The Press release from the client organization, DAFF.

Australia could experience drought twice as often and the events will be twice as severe within 20 to 30 years, according to a new Bureau of Meteorology and CSIRO report.

The article.pdf, a first examination of the data by D Stockwell.

Therefore there is no credible basis for the claims of increasing frequency of Exceptional Circumstances declarations made in the report.

An online opinion piece, by Ian Castles

The recent CSIRO/BOM ‘Drought Exceptional Circumstances Report’ was accepted by government with no external scrutiny: public policy should be made based on this?

Another online opinion piece by Ian Castles.

On July 6, 2008 Prime Minister Kevin Rudd told viewers of the ABC Insiders TV program of the “very disturbing” findings of a study by CSIRO and the Bureau of Meteorology, including that “when it comes to exceptional or extreme drought, exceptionally high temperatures, the historical assumption that this occurred once every 20 years has now been revised down to between every one and two years.”

John Theon

In the last episode, retired NASA Atmospheric Scientist John S. Theon, who claimed to be one of Hansen’s former supervisors, declared himself a skeptic and said Hansen had “embarrassed NASA.” Left leaning blogs cried foul. This week, James Hansen says he doesn’t recall Theon, and John Theon responds that Hansen is losing his memory. From CO2skeptic.

Hansen’s reportedly responded to Theon via email to M.J. Murphy of BigCityLib blog and wrote the following on February 5, 2009:

Hansen wrote: John Theon never had any supervisory authority over me. I remember that he was in the bureaucracy at NASA Headquarters, but I cannot recall having any interactions with him. His claim of association is misleading, to say the least. What he can legitimately say is that he had a reasonably high position in the Headquarters bureaucracy. A job in that bureaucracy is not considered to be a plum, so we should probably be grateful that somebody is willing to do it, and I don’t particularly want to kick the fellow around. You should investigate his scientific contributions to evaluate the degree to which his opinions might be listened to. Of course you are free to quote me. – Jim Hansen – End Hansen Email

Theon fired back at Hansen. “It is absurd that Hansen denies ever meeting me. We have met on numerous occasions. This just demonstrates that Hansen has a poor memory,” Theon wrote on February 5, 2009. [Note: Theon’s complete email response is reprinted below]

“I worked with Hansen from about 1983 to 1994 during which time he was at GISS in NYC and I was at NASA HQ in Washington DC. I retired from NASA in 1995. I had completed 37 and 1/2 years of federal service (civilian Navy, USAF, and including 33 years with NASA.) The money came through me. We were in the Earth Observations Program which later became the Mission to Planet Earth Program. I visited GISS at least once a year to review and evaluate the GISS work. When I visited NYC, to review the research that GISS was funded to do out of the program for which I was responsible, Hansen was most cordial. When I asked him to give a lecture in Japan, he complied,” Theon wrote.

“It was what it was, and no amount of denial will change that,” Theon explained. “I repeat what I wrote to you in January: “I was, in effect, Hansen’s supervisor because I had to justify his funding, allocate his resources, and evaluate his results. I did not have the authority to give him his annual performance evaluation,” he added.

Theon also noted the attacks on him by many of the websites devoted to smearing anyone who questions claims of a man-made climate catastrophe.

“Regarding some of the other attacks that have been aimed at me: I am truly appalled at the backbiting, vitriol that is sent by people who have nothing better to do than try to smear other people’s reputations because they do not agree with their own thinking. To them, I recommend that they get a life,” Theon concluded.

The Dark Side of the Street

Some stories these days send shivers down my spine, over the sheer magnitude of the changes going on in world finance. King Among Clowns is packed with great quotes, about the Congressional testimony of a unlikely anti-hero, with an unlikely name, Harry Markopolos.

markopolos

He ran the numbers. They didn’t fit. He spoke out. See the pillars of power tremble at his words.

Continue reading The Dark Side of the Street