Well I need some stinkin' bar graphs but I need to design them.

There is a problem with monthly data sets being prepared at BOM and that problem is missing data. It's almost as if those with something to contribute to method need to get together to resolve an agreed in fill method that is transparent and portable. In filling does not change the sense of the data much, mostly, but it makes working with the data far more tedious. It also encourages multiple users to devise their own infills, some of which are not reproducible by others. So there are increasingly more versions getting around in the literature and not all of them can be correct.

Yes, I'd very much like tabulation of monthly Tmax and T min at as many stations as practical.

The usual approach is to define level 0, level 1, etc data according to the degree of manipulation. Creating contiguous data ups the levels.

What KMNI has done is really great. Need more of it. Cheers

DAvid, if you want to extract Aus monthly mean data, here's a script that scrapes it. AusInfo is an information sheet (now at CA) listing the 103 stations in the monthly webpages with id numbers. get.bom scrapes the data into a time series using the id (character). I HATE these JAva pages that these guys waste money on.

source("http://www.climateaudit.org/scripts/station/australia/functions.bom.txt")
station= get.bom(id=AusInfo$id[i])

Yeah, we don't need no stinkin' bar graphs.

David, even though BOM says that "the monthly data are not available as a single file because the graphs are generated automatically from the data files and an all-months data file would contain too many data values to fit easily on one graph. We will see what we can do."

I can scrape monthly station data from the BoM website. :)

Here are the first few months from Charters, Qld which I read directly into R:

1 19620101 19620131 28.5
2 19620201 19620228 27.6
3 19620301 19620331 26.1
4 19620401 19620430 22.8
5 19620501 19620531 21.4
6 19620601 19620630 19.4
...

If BoM wishes some assistance, I'd be happy to help them.

Very kind offer Steve. It seems to be taking them a while to work out!

Ben, the daily data for temp as Tmax and Tmin, with year/month/day is already electronic and digital. The hard yards have already been walked by BOM. At most stations it goes back to the year of commencement of records, such as the mid 1850s for Melb and Sydney.

David is talking about the presentation of these figures in a way that makes it easy to inform. Sure, if you are a programmer, they are reasonably easy to manipulate. But it is more efficient for the central controller to manipulate them into a presentation form, than for myriads of public users out there to do their own thing.

As well as data reported on a daily basis, it would be handy to have an official version using in fill of minor missing values by an explained algorithm, followed by tabulation of monthly and yearly averages for each acceptable station. Probably, these are the forms most used in studies.

What is lacking, but is being worked on, is the transcription of meta data such as station changes and other recorded factors that might influence the record.

On the BOM website, the data records tend to commence later. A decision seems to have been made that uncertainties about the start of use of Stevenson screens requires deletion of much data before 1910 or so. Later, there has been a minor trend to report data after about 1950, for reasons that elude me. One day we might see data only after 2000, or whatever was the actual year when the last of the Mercury thermometers was replaced by thermocouple or thermistor devices. It is becoming clearer that Tmax and Tmin from a Hg thermometer is less useful than the Tmax and Tmin from a near-continuous recorder, which can be estimated optionally after smoothing and filtering.

At present, there is no single "official" version of the Australian temperature record, because it is being changed from time to time. This has a bad knock-on effect when other users such as GISS and HadCRU take the same data and do their own adjustments to it. I simply do not know which record to use for studies, because there can be differences in the average of a given year of several degrees C from one source to another.

I suspect that this effect alone has already invalidated some studies that attempted to calibrate proxy data in the instrumented era.

sherro - all good points. Good meta-data and data control is more important than the data themselves. I have worked on this for many years in the museum and zoo communities. Greater appreciation of the needs of rigorous data analysis is one of the causes I want to champion, as I see often where the distribution is aligned around flashy interfaces, rather than substance.

Ben: You are missing the point, though perhaps I didn't explain it clearly. The website is set up for pretty pictures, not with numerate analysis in mind. If you don't give feedback, then they don't know they are out of step with every major climate data provider in the presentation format of the data. Its like no-one else has ever asked for it.

While it is annoying to get, they did provide all the data. It is quite likely that they have no html coders on site (most people don't) and are paying a contract webmaster for updates, so fulfilling your request may end up costing them extra money.

While your request wouldn't be unreasonable if you were their employer, it isn't a reasonable demand that they make it available in a way more convenient to you.

Andrew, the monthly and seasonal data is probably not on the computer before 1950. Do you want to pay someone (even an intern) to type in decades worth of data from handwritten documents (maybe hand-typed), solely benefiting people outside your organization?

Never attribute to malice anything that has another reasonable explanation. The most common of these are stupidity, laziness, and frugality. This appears to be the last of those three.

We should be praising them for having the data so open and available (unlike seemingly everyone else), not griping like children that it isn't in the most convenient format

First, let me say that I agree with the principle you advocate here, also known as Hanlon's razor. Anybody however could drift from such moorings occasionally. But none of this seems an adequate explanation to me. The US government has Nationwide, Regional, Statewide, and a few Local data sets readily available for mean temperature and precipitation by month, season, and annual (as well as YTD) on a single web page, all going back (except for some local records) to 1895, here:

http://www.ncdc.noaa.gov/oa/climate/research/ca...

And it strikes me as odd that, although one may wish for a little more transparency on NOAA's part, that BoM can't be bothered to even do that much work. Suffice it to say that my opinion of the Australian agency is now greatly diminished relative to NOAA here in the states, which I don't hold in particularly high regard anyway.

If you wish to walk around the block 144 times or more, try finding the Max and Min temps for Cape Grim, as well as the data for their GHG measurements. Some temps are available from 1985 on one BOM disc, and on the BOM online site.

Cape Grim is BOM 091245 and is a designated B.A.P.S., acronym for Baseline Air Pollution Station. It has a record of CO2 that might be compared with Mauna Loa. Given its position on the NW tip of Tasmania, I would expect a bit of CO2 to come from exhausted seagulls as they fight the roaring forties. (Lat -40.7, Long 144.7). In past years I helped explore the coastline further south, beyond Macquarie Harbour where "Grim" is a good word for this World Heritage useless wet desert.

The Australian Weather News coverage is back to 1870 for temp. It is difficult indeed for an amateur to draw a graph of monthly or annual CO2 vs Temperature for this site.

Suggestions are welcomed.

Is there any help at
http://www.australianweathernews.com/

I am not surprised my numerate readers would have no trouble getting the data into the form they require. My point was more along the lines of "Why would I want to walk around the block 12 times to go to the coffee shop when I only have to walk around once?" And why has nobody asked for it to be in a more convenient and standard form for data analysis before?

David,
I found this R scrap seemed to work, returning a 12 col matrix:
s1="http://www.bom.gov.au/web01/ncc/www/cli_chg/timeseries/tmax/01/aus/latest.txt"
for(i in 1:12){
j=62
if(i>9)j=61
substring(s1,first=j)<-as.character(i)
download.file(s1,"temp.txt")
vv <- matrix(scan("temp.txt", 0), ncol=2, byrow=TRUE)
j=dim(vv)[1]
if(i==1)monthly=matrix(0,j,12)
monthly[1:j,i]=vv[,2]
}

Why do they have annual mean data back to 1910 or so, but seasonal and monthly data back to 1950? Something they don't want people to see?

They probably think its a world class interface. It looks very good. But its not what this customer wants, which is a simple link to a text file of all the data.

"So the reason why stock standard monthly data series are not available is because they don’t look good as a barchart."

I guess I have to take you at your word.

HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA

Do these people really understand how poorly they look to those of us looking in??

"To get the data into a monthly series would require me to download all 12 raw datafiles (one for each month) then interleave them to create a continuous list with 12 values for each month. This would be timeconsuming and a potential source of error."

Actually, in Excel at least, this is fairly easy. I would just take each file, turn it into a table:

year, value
year, value

and so on, then put the table in Excel, then move the values column over one, and put a string of ones for January, twos fro February, etc. between the values columns and the years columns.

I would then take each set of three columns and put each set right under the last month's set.

Then, I would select all what is now three columns of 12 sets of the same year, a hundred or so ones, twos, threes, etc, and the values. Then I'd click on the sort button. Bingo, all the years line up, the months are in order, and you're done!