Comments on: CSIRO Data Policy: Go Pound Sand http://landshape.org/enm/csiro-data-policy-go-pound-sand/ The power of numeracy Tue, 09 Mar 2010 14:17:07 -0600 http://wordpress.org/?v=2.8.6 hourly 1 By: Mangled Thoughts » Scoop: Can the CSIRO explain an explosive contradiction? What is going on? http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-179957 Mangled Thoughts » Scoop: Can the CSIRO explain an explosive contradiction? What is going on? Wed, 17 Feb 2010 09:08:33 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-179957 [...] Stockwell-CSIRO exchange: Stockwell, ‘CSIRO Wars’. Mr. Peter Gallagher, summation, to which one of his readers wrties of the same type of mendacious [...] [...] Stockwell-CSIRO exchange: Stockwell, ‘CSIRO Wars’. Mr. Peter Gallagher, summation, to which one of his readers wrties of the same type of mendacious [...]

]]> By: admin http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-168049 admin Mon, 21 Jul 2008 03:28:55 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-168049 I agree with you. The spread of the models is a reflection mostly of uncertainty in the models themselves, knowledge of the system, and not of inherent physical variability. that said, there are results that claim to show that averaging or mixing models leads to improved prediction, but the field of the distributional properties of statistics composed of mixtures of models is an active area of statistical research, see http://arxiv.org/abs/math/0702781. I agree with you. The spread of the models is a reflection mostly of uncertainty in the models themselves, knowledge of the system, and not of inherent physical variability. that said, there are results that claim to show that averaging or mixing models leads to improved prediction, but the field of the distributional properties of statistics composed of mixtures of models is an active area of statistical research, see http://arxiv.org/abs/math/0702781.

]]> By: david http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-168043 david Sun, 20 Jul 2008 22:56:31 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-168043 No, models are fine by me in principle. It's a question of how they are validated. It's obvious to me (so obvious that I keep thinking I'm missing the point, so please tell me if I am) that each model must be validated independently. If model A predicts temperatures that are far too high and model B predicts temperatures that are far too low, we have too failed models. I don't think it means anything at all if the average of the two happens to agree reasonably well with observations. To think it does implies some kind of mechanistic understanding about the overestimation of one model being "corrected" by the other. But if that's the case, surely you would abandon models A and B and incorporate the "correction" to give model C, which is then tested against the observations. I think that doing a significance test is just an extension of that mistake. No, models are fine by me in principle. It’s a question of how they are validated. It’s obvious to me (so obvious that I keep thinking I’m missing the point, so please tell me if I am) that each model must be validated independently. If model A predicts temperatures that are far too high and model B predicts temperatures that are far too low, we have too failed models. I don’t think it means anything at all if the average of the two happens to agree reasonably well with observations. To think it does implies some kind of mechanistic understanding about the overestimation of one model being “corrected” by the other. But if that’s the case, surely you would abandon models A and B and incorporate the “correction” to give model C, which is then tested against the observations. I think that doing a significance test is just an extension of that mistake.

]]> By: admin http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-168040 admin Sun, 20 Jul 2008 21:20:32 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-168040 It is risky to trust models too much, unless they have been throughly validated be and based in solid physical knowledge. Wouldn't you be throwing out all models in your view. It is risky to trust models too much, unless they have been throughly validated be and based in solid physical knowledge. Wouldn’t you be throwing out all models in your view.

]]> By: david http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-153219 david Fri, 18 Jul 2008 08:07:31 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-153219 But isn't it risky (if the aim is to keep these guys honest about testing their models) to allow that such a test is even in principle valid? The way I see it is that it would be possible to pick one of the model parameters, say p, and run simulations such that a selected output (say, predicted temperature in 2050) is some single-valued function f(p) of p. Now, if I find out what statistical test you intend to do then I as the modeller can produce a set of "observations" f(p(i))(model outputs) that can either pass or fail it, at will. In doing that I would obviously be open to major criticisms on other grounds (why those p(i)?) but given the current climate of debate I would easily get away with playing the card: you attacked me on statistical grounds and I won, and now you are trying a completely different tack on dynamical grounds. So when I win that, what next? Personal grounds? Typical Big Oil. But isn’t it risky (if the aim is to keep these guys honest about testing their models) to allow that such a test is even in principle valid? The way I see it is that it would be possible to pick one of the model parameters, say p, and run simulations such that a selected output (say, predicted temperature in 2050) is some single-valued function f(p) of p. Now, if I find out what statistical test you intend to do then I as the modeller can produce a set of “observations” f(p(i))(model outputs) that can either pass or fail it, at will. In doing that I would obviously be open to major criticisms on other grounds (why those p(i)?) but given the current climate of debate I would easily get away with playing the card: you attacked me on statistical grounds and I won, and now you are trying a completely different tack on dynamical grounds. So when I win that, what next? Personal grounds? Typical Big Oil.

]]> By: admin http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-146847 admin Thu, 17 Jul 2008 03:11:01 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-146847 Yes, they are model predictions. I contend even tests on the predictions are non-significant. Yes, they are model predictions. I contend even tests on the predictions are non-significant.

]]> By: david http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-146838 david Thu, 17 Jul 2008 02:07:37 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-146838 Am I right in thinking from this: " For the frequency (temp and rain), the sample is the number of models (13)" that the statistics are being carried out on model predictions, not observations? If so, there's no way this could be regarded as a random sample and a statistical test would be irrelevant. It would all depend on the systematic biases in the individual models (which are presumable deterministic). Am I right in thinking from this:

” For the frequency (temp and rain), the sample is the number of models (13)”

that the statistics are being carried out on model predictions, not observations? If so, there’s no way this could be regarded as a random sample and a statistical test would be irrelevant. It would all depend on the systematic biases in the individual models (which are presumable deterministic).

]]> By: admin http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-145943 admin Wed, 16 Jul 2008 02:36:23 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-145943 Ian, Thanks for your background information on this and CA. It all points to the conclusion that working in the global warming area is about policy, litigation and quasi-litigation and the scientific mores we were fond of are relics of bygone era. Ian, Thanks for your background information on this and CA. It all points to the conclusion that working in the global warming area is about policy, litigation and quasi-litigation and the scientific mores we were fond of are relics of bygone era.

]]> By: Ian Castles http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-145938 Ian Castles Wed, 16 Jul 2008 02:06:44 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-145938 David, I agree with your comment on Climate Audit that Kevin Hennessy could be acting on advice in invoking Intellectual Property Rights in order to avoid providing you with the data required to verify the statistical significance of the CSIRO findings ( www.climateaudit.org/?p=3267 ) In an email of 16 March 2006 I advised Kevin Hennessy of an error in a paper of which he had been a co-author (Whetton et al, December 2005, “Australian climate change projections for impact assessment and policy application: a review”, CSIRO Marine and Atmospheric Research Paper 001). The paper stated that the IPCC scenarios were ‘deliberately constructed to be equally plausible’ (p. 33). I was pleased to receive Kevin’s response on 29 March in which he wrote “I agree with you [Castles]. The error will be corrected”. This email was copied to seven of Kevin's CSIRO colleagues: Greg Ayers, Chris Mitchell, Penny Whetton, Ian Watterson, Kathleen McInnes, Paul Holper and Simon Torok. After more than two years, the error has still not been corrected. I think that the reason for CSIRO’s unwillingness to do this is that the paper in which the statement appeared had already been cited as “In prep.” in the First Order draft of the IPCC Report, circulated on 13 August 2005. This makes it difficult for the CSIRO to admit its error. On 11 October 2006, Roger Jones (another co-author of the CSIRO Research Paper, and another IPCC Coordinating Lead Author) replied as follows to an inquiry I’d made about CSIRO’s progress in correcting their error: “Your [Castles] point about the error in our paper made earlier this year was noted. A request was immediately sent by a colleague through to the relevant people to correct and repost the document after you first pointed this out but it was not done (a production task). On the basis of your recent post pointing out that the phrase had still not been corrected (quelle horreur) we have asked that it be followed through. There is no conspiracy - it was a breakdown in process.” (John Quiggin’s blog, “Drying Out” thread). It seems that either (a) the CSIRO editing process has continued to break down for the succeeding two years or (b) it is CSIRO’s policy not to correct errors. I think that the latter is more likely. David,

I agree with your comment on Climate Audit that Kevin Hennessy could be acting on advice in invoking Intellectual Property Rights in order to avoid providing you with the data required to verify the statistical significance of the CSIRO findings ( http://www.climateaudit.org/?p=3267 )

In an email of 16 March 2006 I advised Kevin Hennessy of an error in a paper of which he had been a co-author (Whetton et al, December 2005, “Australian climate change projections for impact assessment and policy application: a review”, CSIRO Marine and Atmospheric Research Paper 001). The paper stated that the IPCC scenarios were ‘deliberately constructed to be equally plausible’ (p. 33). I was pleased to receive Kevin’s response on 29 March in which he wrote “I agree with you [Castles]. The error will be corrected”. This email was copied to seven of Kevin’s CSIRO colleagues: Greg Ayers, Chris Mitchell, Penny Whetton, Ian Watterson, Kathleen McInnes, Paul Holper and Simon Torok.

After more than two years, the error has still not been corrected. I think that the reason for CSIRO’s unwillingness to do this is that the paper in which the statement appeared had already been cited as “In prep.” in the First Order draft of the IPCC Report, circulated on 13 August 2005. This makes it difficult for the CSIRO to admit its error.

On 11 October 2006, Roger Jones (another co-author of the CSIRO Research Paper, and another IPCC Coordinating Lead Author) replied as follows to an inquiry I’d made about CSIRO’s progress in correcting their error:

“Your [Castles] point about the error in our paper made earlier this year was noted. A request was immediately sent by a colleague through to the relevant people to correct and repost the document after you first pointed this out but it was not done (a production task). On the basis of your recent post pointing out that the phrase had still not been corrected (quelle horreur) we have asked that it be followed through. There is no conspiracy – it was a breakdown in process.” (John Quiggin’s blog, “Drying Out” thread).

It seems that either (a) the CSIRO editing process has continued to break down for the succeeding two years or (b) it is CSIRO’s policy not to correct errors. I think that the latter is more likely.

]]> By: John Baltutis http://landshape.org/enm/csiro-data-policy-go-pound-sand/comment-page-1/#comment-145532 John Baltutis Tue, 15 Jul 2008 19:45:46 +0000 http://landshape.org/enm/csiro-data-policy-go-pound-sand/#comment-145532 Maybe you should be asking the DOE's Office of Science for the modeling dataset. From the report's acknowledgements: <i><b>We acknowledge the modelling groups, the Program for Climate Model Diagnosis and Intercomparison (PCMDI) and the WCRP's Working Group on Coupled Modelling for their roles in making available the WCRP CMIP3 multi-model dataset. Support of this dataset is provided by the Office of Science, US Government Department of Energy. </b></i> Maybe you should be asking the DOE’s Office of Science for the modeling dataset.

From the report’s acknowledgements:

We acknowledge the modelling groups, the Program for Climate Model Diagnosis and Intercomparison (PCMDI) and the
WCRP’s Working Group on Coupled Modelling for their roles in making available the WCRP CMIP3 multi-model dataset.
Support of this dataset is provided by the Office of Science, US Government Department of Energy.

]]>