Nonstationary processes have been defined as random processes whose statistical properties vary over time (see Stationary and Nonstationary Random Processes by Michael Haag).

Nonstationary statistics presents a number of problems. For example, the definition of the power spectrum is a problem, as variation in properties is similar to low frequency noise.

The following is the most succinct statement of the problem of analysing non-stationary statistics I have seen. The following is quoted directly from comments by ‘bender’ starting in comment #18 in response to the post More Tangled Webs at Steve McIntyre’s blog ClimateAudit.

A high AR1 coefficient does not necessarily imply a trend+low AR1 coefficient. However a trend+low AR1 coefficient does lead to a high AR1 coefficient. That is the one of the points these guys are making: AR1 models are an improvement over AR0 models, but they are fraught with their own problems.

The problem with these Hockey Stick-shaped series is that they are nonstationary, so AR coeffs do not have a straightforward interpretation. (Split any time-series at the join of the shaft and blade, compute the PACF and you will see what I mean when you compare the two.) You could take out the trend, to give the coeffs a straightforward interpretation, but then you’ve got the problem of interpreting what it is you’ve taken out, and an autocorrelation analysis certainly isn’t going to help you now.

The purpose of autoregression is to figure out how Xt varies as a function of Xt-1. If they are autocorrelated only indirectly, through the action of some other forcing variable, then the autoregressive model is a bad model, and this badness will revealed when the forcing agent fades in and out (as teleconnections are wont to do).

Bender continues in post #20.

1. For a quasi-demonstration of how a PACF changes when a trend is removed, compare the PACFs of the tropical storm count (with 1970-2005 trend) vs. the landfalling hurricane count (without trend) with which it is strongly correlated (r=0.62 before 1930, r=0.49 afterward). See how PACs 1-4 drop in the detrended series?

2. A clarification for anyone who finds it necessary: the purpose of autoregression is to identify endogenous processes that are persistent through time. Exogenous processes that fade in and out tend to inhibit the estimation of the endogenous autoregressive component, because they introduce a complex nonstationary noise structure.

3. Incidentally, the more dominating the low-frequency exogenous component(s), the lower the precision on the ARMA model estimates. This is the real problem with 1/f noise: you increase your sample size over time, and you inevitably uncover some new “trend” caused by some hitherto unknown exogenous forcing agent. Consequently, it is impossible to obtain an “out-of-sample” sample. (Your new samples come from different populations, which thus nullifies the validation test.)

Thanks Bender…