Table of contents for CSIRO/BoM DECR

  1. Temperature Index Drought
  2. Comparison of Models and Observations in CSIRO/BoM DECR
  3. Western Australia Future Rainfall
  4. DECR: The message starts to slide
  5. Scientists Biasing Research

There must have been some way that the models of exceptionally low rainfall (drought) were validated in the CSIRO/BoM Drought Exceptional Circumstances Report. Usually, models are checked against observations to make sure they have ’skill’ at the purpose for which they are intended. In this case, the global climate models used in the Drought Exceptional Circumstances Report must have been compared with observations, before they were used to show increasing frequency and severity of droughts in the next 30 years.

In Figure 10 of the report (above right), the exceptionally low rainfall observations are plotted against model projections. However, the figure is more of a cartoon and hard to read, so I replotted it using data from the report provided by Kevin Hennessy (above left). There are a number of other difficulties with Figure 10 that make it hard to see miss-match between the models and observations. The y axis is a large 30% exceptional low rainfall area, the drought observations are averaged over 10 years instead of the usual climatological average of 30 years, and the lines of the model are different again, with a jagged appearance. The confidence intervals are not standard deviations, but yet another novel metric. In replication of their figure 10, using R for the statistics, the last panel shows data for all regions.

On the figure in red are the 30 year moving average of observed percent area with exceptionally low rainfall. In black with dashed confidence intervals (1 s.d.) are the averages of the 13 models used in the study. This is the same data as figure 10.

It looks to me that in the last half-century of observations (1950-2007) in almost all regions droughts are decreasing (red), while the models show drought increasing (black). SW-WA and VicTas are a little different as the observations are constant, while models are increasing.

These graphical observation are consistent with the numerical results in Table 1 of the report Tests of Regional Climate Model Validity in the Drought Exceptional Circumstances Report that the regions SW-WA and VicTas are the only regions where the models of drought are not of the opposite sign to the observations.

So we conclude that while the observations of drought are decreasing in the last 50 years (in terms of a climatological average of frequency and areal extend of exceptional low rainfall), the projections of drought are increasing over the same period. This is supported by the last panel in the figure where all regions are averaged, the models predict increasing droughts, but the observations show decreasing droughts. The models go in the opposite direction to observations.

I have been in contact with Kevin Hennessy a number of times about the report, but unfortunately cannot report much progress. He maintains that the authors were satisfied with the validation of the models in the report, but has not provided details of the validation procedures or results that they used. Unfortunately the validation of the models was not reported in the DECR either. The only reference I can find is in Section 4.3 where it states:

The observations are generally within the range of individual model results.

It seems like the ’skill’ they are thinking of is in the range of model results enclosing the observations (generally). But projections of linear regressions are the more usual way of projecting model results, and more statistically well defined. Moreover, their statement above is not quantitative, and is done by what is known as ‘eyeballing’. It is not a validation test, per se.

As the data provided by Kevin Hennessy as used in the report were obtained after considerable pressure from a number of blogs across the web, originally withheld due to ‘Intellectual Property’ reasons, I was hoping he would be more forthcoming this time.

Looking at the forward projections of the models, droughtedness increases in SW-WA, SWAust and VicTas, but decreases in the other models. Given that the trend of models don’t resemble the observations at all in the 1950-2007 period, I think little confidence can be put in these forward projections. It is very strange to find models performing so differently to observations, and yet the projections of the consequences of warming to be stated with such certainty, as in the quote from the summary (below). Note also that the regions where model projects decreasing droughts are interpreted as ‘little detectable change’, suggesting strong interpretive bias in order to create alarm over increasing droughts.

If rainfall were the sole trigger for EC declarations, then the mean projections for 2010-2040 indicate that more declarations would be likely, and over larger areas, in the SW, SWWA and Vic&Tas regions, with little detectable change in the other regions.

I am very curious to see the methods of validation they used and the actual results they obtained. I would also like to know why the results of the model validation were not reported.

Previous in series Next in series