Niche Modeling

The power of numeracy

Niche Modeling header image

Getting Advantage for Analysts from Policy

September 12th, 2008 by admin · 9 Comments

Below my reply to the letter here from Andrew Ash, Director of the Climate Adaptation Flagship. Concerning the Drought Exceptional Circumstances Report (DECR) there was an earlier more technical draft that could have contained validation information, but was returned by the client (DAFF) because it was unsuitable for stakeholder policy work. What does that mean? The earlier report should show what/if validation was performed to determine if the models had skill at predicting drought, and what changes were made to the report to make it palatable for the policy wonks, that is, sounding more like a disaster movie.

10 Sep. 08

To:
Andrew Ash (Director, CSIRO Climate Adaptation Flagship)
Gary Foley (Acting Director, Australian Bureau of Meteorology)

Dear Sirs,

Re: Request a copy of first draft of DECR

Thank you for your prompt consideration of my letter of 3 Sep. expressing my concern with the validity of key claims in the Drought Exceptional Circumstances Report (DECR).

I strongly share your sentiment “as you know we value constructive criticism as it often helps to tighten up that science, which is vital in issues which matter so much to the future of our planet.” This leads me to believe you would be amenable to providing a copy of the first (more technical) draft of the report, in order to provide constructive criticism of the validation performed on the models used in the DECR.

We also seem to be in substantial agreement on the difficulty of modelling exceptionally low rainfall years – the substance of my critique:

1. Kevin Hennessy stated as much in the CSIROpod interview of 15 Jul.

(1.20m) There’s not been a clear indication of changes in exceptionally low rainfall years, and that’s because there’s lots of variability between decades …
(1.40m)But in terms of a long term trend its not very clear in terms of exceptional low rainfall years.

2. This is the basis of my claim in the previously supplied “Tests of Regional Climate Model Validity in the Drought Exceptional Circumstances Report”.

3. You do not dispute this point in your letter of 9 Sep.

However, I respectfully submit it is not the case that my critique concerns only a narrow component of the report:

1. Under the terms of reference the BoM and CSIRO were requested (DECR Appendix 1), on the basis of current knowledge of climate change science, to assess: (points 2 and 4 of 4 points):

2. Likely changes in the nature and frequency of severe rainfall deficiencies over the next 20-30 years, in comparison to severe rainfall deficiencies defined by the available instrument records. Severe rainfall deficiency is defined as an event in the lowest 5th percentile of historical records persisting for prolonged periods and over significantly sized regions.

4. The place of past exceptional climatic events in the context of the likely frequency and severity of future climatic events.

2. Projections of exceptionally low rainfall using models constitute roughly one third of the substance of the report, along with temperature and soil moisture.

3. The dictionary definition of drought is: a shortage of rainfall. If my concerns are as narrow as to not affect the overall report, then I would respectfully suggest the report is misnamed, the word “Drought” should be dropped and replaced with “Temperature Extreme Circumstances”, or something similar. This would be consistent with the summary of the findings at the CSIROpod interview site, which only mentions exceptionally hot years, and omits claims of exceptionally low rainfall.

4. The claims at issue were quoted by the Prime Minister, the client DAFF, and in the media.

I submit that in a report whose terms of reference were to determine “likely changes in the nature and frequency of severe rainfall deficiencies over the next 20-30 years”, my critique of a major part of the summary regarding the likelihood of exceptionally low rainfall does address the main thrust of the report.

Kevin Hennessy in an email of 11 Jul. (excerpt below) mentioned the existence of a draft of their report with a great deal more technical information.

Thirdly, the Terms of Reference (Appendix 1) state that “it will be presented in a form that will enable it to be used in future drought policy discussions, including stakeholder consultation”. Our first draft of the report was considered too technical by the client (DAFF), since the target audience is for lay-people, so we had to spend considerable time simplifying the language, diagrams and tables. Therefore, statistical tests and results from individual climate models were not presented.

Kevin indicated in an email of 25 Sep. that:

… we are satisfied with the model evaluation done prior to publication.

I request a copy of the first draft of the DECR, commercial-in-confidence if necessary.

Sincerely,

David R.B. Stockwell PhD

10 September 2008

Tags:

9 responses so far ↓

  • 1 Peter Gallagher // Sep 13, 2008 at 1:07 am

    So far, CSIRO has shown a disturbing reluctance to discuss the data and reasoning behind their report. This fluffy correspondence about their good intentions is obviously insufficient. If they want to demonstrate that they “value constructive criticism” then they should respond to your letter in a substantive way.

    If they do not, then their models do not deserve our confidence and the Prime Minister will be left with no substantial basis for his alarming claims.

    I look forward to hearing that careful science, not just spin, lies behind the ‘public’ version of the DEC report. Or, if necessary, that a respect for science will lead CSIRO to revise their report in the light of valid criticism.

  • 2 admin // Sep 13, 2008 at 1:51 am

    Hi Peter, Like most people, I can tell when people are blowing smoke. Next stop, FOI, I think.

  • 3 Steve Short // Sep 13, 2008 at 1:57 pm

    Best of luck. From in-depth personal and painful experience I can say that to pursue technically detailed FOI applications with Federal Govt. departments can be to unwittingly enter a strange and drawn-out twilight zone which drifts between states alternately resembling parts of a Franz Kafka novel to excepts from Alice and Wonderland to acts of a Gilbert and Sullivan opera. Be prepared to accept that your chances of success are very low. Almost 12 years of having Rumpelstiltskin sitting at the top of the Canberra muck heap had a very deep effect.

  • 4 Ian Castles // Sep 14, 2008 at 3:12 am

    Steve,

    My personal experience is exactly the opposite. In June 2007 I made an FOI request of the Commonwealth Department of Agriculture, in the hope of obtaining information which would shed light on the IPCC’s treatment of my efforts over several years to bring more rigour to the Panel’s economic work, particularly in relation to emissions scenarios.

    I received in response a 5 cm thick file of papers which answered most of my questions. In my view, these papers constitute a devastating indictment of the IPCC process.

    The Department denied access to some relevant papers. For example, they declined to provide emails sent by the Chair of the IPCC, Dr Rajendra Pachauri, to the Executive Director of ABARE, Dr. Brian Fisher, between 13 December 2002 and 13 September 2006 on the grounds that ‘Disclosure of the document would divulge information communicated in confidence by or on behalf of an international organisation (namely the Intergovernmental Panel on Climate Change) to the Government of the Commonwealth’ (s. 33, FOI Act). But I already know what Dr Pachauri told journalists about me at international meetings in Paris (February 2003) and Milan (November 2003), and I’m not particularly interested to discover what further defamatory comments he may have made about me in communications made in confidence to the Australian Government.

    Now for some examples of the information I DID receive in response to my FOI request. On 25 February 2003, Brian Fisher wrote in the following terms to Dr Nebojsa Nakicenovic, who had been Coordinating Lead Author of the IPCC Special Report on Emissions Scenarios:

    ‘Dear Naki,

    ‘I am writing to invite you to take part in an international climate change meeting that ABARE is hosting in Canberra on 7 and 8 May 2003. The focus of the meeting will be on the role of modeling analysis in climate change policy.

    ‘ABARE and a number of US collaborators [three US institutions were named] are organising the meeting to bring together a small number of key influential modelers and policy makers/negotiators from around the world …

    ‘Participants from a range of developed and developing countries are being invited. Reaction so far from the policy community has been very positive, with acceptances from … Japan, … China, … the United Kingdom …, Columbia, … India and … Canada [the participants were named] … Dr Rajendra Pachauri (Chair, IPCC) has accepted our invitation …

    ‘A funding package covering a return airfare to Canberra and accommodation during the meeting will be available to support your attentance …’

    On 15 March Dr Nakicenovic replied, in part, as follows:

    ‘Dear Brian

    ‘I am sorry for the long overdue response. The reason is first that I could not open the attachment while I traveled and then had to prepare under a lot of stress a rebuttal to Castles-Henderson special issue of Energy and Environment where they are publishing their collected works from the [Lavoisier Group] website …’ [Note: The ‘collected works’ consisted mainly of letters written to Dr Pachauri at his invitation and the presentations that David Henderson and I made, at the Panel’s invitation, to the IPCC Expert Meeting on Emissions Scenarios in Amsterdam in January 2003. With my agreement, Brian Fisher was invited to the Amsterdam meeting and participated in the associated discussions about the Castles and Henderson critique. IC].

    And on 18 March Brian Fisher wrote again to Dr Nakicenovic:

    ‘Naki, I certainly hope that you can reorganise your classes so that you can come to Canberra. I’m sorry to hear that you’ve been stressed by the Castles-Henderson saga. Unfortunately, I think it will continue and that the only solution is to continue to present rational arguments. Please keep in touch on the workshop and on anything you think I might be able to help you with on Castles-Henderson. All the best, Brian.’

    The workshop duly took place at The Boat House by the Lake, Canberra on 7-8 May 2003. The papers provided to me by DAFF include the final program and the list of participants (51 in all). There were 30 Australian participants, of whom 29 were from departments and agencies of the Commonwealth Government. There were also 13 participants from other governments, including representatives from the US, the UK, India, Japan, Canada, Korea, New Zealand and Norway. Representatives from each of the three US sponsoring institutions participated and made presentations. The Chair of the IPCC, Dr Pachauri, was the discussion opener for the session ‘How can analysis best contribute to future climate negotiations and policy responses?’

    Significantly, I was not invited to the workshop and THERE WAS NO PARTICIPATION FROM THE AUSTRALIAN TREASURY OR THE AUSTRALIAN BUREAU OF STATISTICS - notwithstanding that the Australian Government had urged in its submission to the IPCC on the scoping of the Panel’s Fourth Assessment Report (March 2003) that:

    ‘It will be important to draw upon a broader range of economic and statistical expertise in the development and explanation of the economic aspects of the emission scenario content of the AR4. Specifically, the second scoping meeting of the AR4 should engage a broader range of experts and ensure representative participation is sought at a national level from the economics, statistical, and economic history professions, and include representation from key international bodies such as the United Nations Statistical Commission.’

    Whatever the intentions of ABARE and the three US sponsors of the workshop, the result was that the IPCC and the US Climate Change Science Program have been able to persist with the discredited approach of using exchange rate conversions to measure the relative sizes of national economies. In April 2005 five economists at ABARE (including the Coordinating Lead Author and the only other Australian Lead Author of the IPCC chapter in which the criticisms of the SRES were reviewed and rejected) presented a paper to an APEC workshop in Korea that included the following:

    ‘China … is forecast to overtake Japan as the second largest economy in APEC at about 2028. By 2030, China is expected to be just over a quarter of the size of the US economy’ (Matysek et al, 2005, The Nexus between Climate, Energy & Technology, APEC Business and Climate Cjamge and 2nd Asia Region Climate and Energy Workshops, Seoul, Korea, 11-13 April 2005).

    Does ANY recognised expert believe this any more, even among the IPCC milieu? This is an entirely false perspective on the world that we live in, and the habits of thinking that it represents have gravely misinformed policymakers.

    I’m happy to make the response to my FOI request available for examination and report by an appropriate independent body (e.g., a university public policy or political science department). Such a report would be especially enlightening to anyone who might otherwise have given credence to the extravagant claims made about the IPCC by such opinion-makers as Professors Stephen Schneider or Clive Hamilton (Professor of public ethics at the Centre for Applied Philosophy and Public Ethics).

    In ‘The global warming debate - Professor Stephen Schneider’s response to Professor Don Aitkin’ (Ockham’s Razor, ABC Radio, 18 May 2008, http://www.abc.net.au/rn/ockhamsrazor/stories/2008/2249809.htm), Professor Schneider claimed that ‘With nearly 4,000 people participating from over 130 countries, each new report is probably the most scrutinised scientific document in the world.’ And in ‘Death rattles of the climate change skeptics’ (New Matilda, 19 May 2008, http://newmatilda.com/2008/05/19/death-rattles-climate-change-skeptics ), Professor Hamilton describes the IPCC process as ‘painfully exacting and comprehensive.’

    It is not so.

  • 5 Steve Short // Sep 14, 2008 at 4:41 am

    Ian

    I am pleased for you. My experience was exactly the reverse. It is not appropriate for me to detail my experience here as it was for an intensely personal reason and you could read about it (briefly) on Peter Gallagher’s blog if you wished. Suffice to say that a team comprising a top Qld barrister, a retired senior Federal Govt. bureaucrat (with Uni. Syd. law degree) and I (as their client) could not obtain a fair and equitable release of technical documents out of Aviation Australia and CASA despite the most detailed of efforts to AAT level.

  • 6 Ian Castles // Sep 14, 2008 at 5:04 am

    Thanks Steve. I’m not suggesting that my experience was typical, but it does show that FOI requests can sometimes yield illuminating information. And it is particularly relevant in this case because DAFF, which was reasonably forthcoming in response to my request, presumably has a copy of the document that David is seeking (the first draft of the DECR).

    If the authors and CSIRO and BoM aren’t prepared to release this document, maybe the client department will. I think it would be worth a try.

  • 7 davids // Sep 14, 2008 at 7:18 am

    Ian, do you want me to put this material on the web? I can put the organized skeptic movement to work and scan it 8)

  • 8 Ian Castles // Sep 14, 2008 at 8:24 am

    Thanks for the offer David. As I noted, the material is voluminous and I think it would be hard going for readers (less familiar with the context than I am) to distil the relevant parts if the material was just scanned and placed on the web ‘as is’ - a possible end-result could be the sort of interminable argument that’s developed over the NAS and Wegman reports, with some commentators focusing on the substance of the issues and others focusing on what it all says about the IPCC process. Overall it seems to me that the enormity of the IPCC’s failure in making so much of the original MBH papers may have been obscured during this protracted process (though future researchers will be ever-grateful for the resources now available on the CA website).

    That said, I should make clear that what I have quoted above is only the tip of the iceberg. ABARE organised two more ‘Boat House’ workshops - one in India in March 2004 and one in Paris in April 2006. The whole exercise was almost a ’shadow’ IPCC Working Group III operating secretly, with the ‘official’ WGIII working in tandem and archiving its proceedings (inadequately) on the IPCC website.

    This is the line of thinking that led me towards the suggestion that the material be examined by an independent body, but I can see the merit in having the whole file scanned and put into the public domain.

    Another aspect that I wonder about as I write this is whether the material is covered by copyright, with reproduction not permitted.

  • 9 admin // Sep 14, 2008 at 8:33 am

    Ian, It would be indexed by google promptly on my site, though setting up sections would also be of value.

    That is interesting about the Boat House workshops. I am working on a post to the effect that Garnaut represents an ‘independent’ alternative to areas of the IPCC and has come out on the skeptics side. The point is that IPCC is overrated, its just another review, and should be considered alongside other reviews. It would be good if this perspective could be made available via improved access to documents somehow.

Leave a Comment