“Linear” regression with a formula y = a + bx + cx2 + dx3?

I was taught that “linear” relationships had to have a formula based on y = mx + c. Surely you mean “cubic” regression, or am I out of date?

Musing aloud

Suppose you took this or a much longer series (to increase the probablility of repeat patterns of temp) and worked a daily basis to get ever more numbers. Like Melbourne 1860-now.

Then, you do the above exercise increasing the powers to quadratic, 5th, 6th, etc. Do you think that you would head off to needing a Guide to the Galaxy, or do you think that some of the components would start to come back towards the curve? I know there are more conventional ways to attack this, but I’ve never seen the exercise done.

Patrick: Ah ha, I am glad you asked. No they are called linear regressions because they are linear in the terms. As opposed to non-linear like y=a+bxz. The formula you show is a ‘simple’ linear regression.

Geoff: You would have to try it. Why?

David, I have to agree with Patrick there; it’s cubic regression in x. It’s true that it is linear in each of the three variables x, x^2 and x^3, but these aren’t independent.

As you add powers, as Geoff suggests, you run into the difficulty that they all start to behave rather similarly, so it is difficult to discriminate the effects of each one. That is why people recommend fitting orthogonal polynomials (eg Chebyshev). You still just get a polynomial, but the coefficients are more meaningful.

From Wikipedia:

For example, the model is linear in the parameters, and , but it is not linear in x, as , a nonlinear function of xi.

David,
On re-reading your post, I can’t see the point of it at all. When you fit a cubic, the only claim is that the cubic fits. No-one expects the terms added to make the cubic polyniomial will fit. What do you expect them to do?

Nick, Its an interesting pathological example. I don’t know about ‘no-one’. The same thing can happen when the terms are not higher order. The point is that just because the significances are all high in linear regression, the individual terms cannot reliably be ‘attributed’ to anything.

People attribute the terms of multiple regressions all the time, interpreting the significance of the variables as meaning something. The conditions under which the terms of a regression can be attributed to something are more strict.

David,
There’s nothing pathological about it. I repeat my question – how do you think the components should behave?
If you move the origin to the centre point, you’ll get components with much less apparent variation (but exactly the same cubic) If you move it to, say, 50 years, you’ll get components looking much less like the final curve (but again, the same cubic).

Well pathological in the sense that if you were expecting the multiple regression to return sensible values for the coefficients, eg. small positive linear term, and very small non-significant square and cubic terms, you would be disappointed.

You would be misled if you were to conclude from this fit that there was a couple of strong exponential factors ‘forcing’ global temperatures.

Obviously the linear regression gives the ‘right’ answer subject to its assumptions. The point is that the results that it produces are not always meaningful and can be misleading.

David,
The lack of independence of the powers leads to apparently high significance which may be misleading. When you quote the coefficient d as -1.042e-04 with a std error of 1.837e-05, you test the significance of varying d keeping other coefficients fixed. But if you tested the significance of d when varied letting other coefficients take their corresponding fitted values, it would be much less. That is one reason why using orthogonal polynomials is better.

Again, as a check, try doing the analysis just subtracting 15 from all the x values.

My reasoning in post 2 is that the component curves disappear off the graph with potentially increasing speed as the power is raised. Thus, the composite final involves the subtraction of large numbers (which as an aside raises Q of significant figures in the calculations). I merely wished to see if there was a power, however high within reason, that had a very small component or coefficient and so did not streak off towards infinity so fast. Then I would wonder again of the components did have some significance other than numeric, because of the result being unusual and seeking an explanation.

The quote from Wikipedia means that if the expression is not linear in x then there is not a linear regression in x. How can you not understand that? It seems pretty simple to me.

Why not call it a cubic regression? Or do you think that be a cubic regression in x you would need cubic variables in the coefficients of x?

Geoff. I have posed some more variations of powers above.

Patrick, polynomials of the form above are referred to a linear regressions in statistics, despite being not linear in x. The reason I think is because the solution is via basic linear algebra, inversion of a matrix. When the equation is non-linear then linear algebra won’t work.

David,
I ran your R script, just renumbering the years so year 15 was set as year 0. The table of coefficients printed out is now:

Estimate Std. Error t value Pr(>|t|)
(Intercept) 7.288e-02 1.349e-02 5.404 1.21e-07 ***
x 3.071e-02 2.630e-03 11.676 2e-16 ***
I(x^2) 1.017e-04 1.381e-04 0.736 0.462
I(x^3) -1.042e-04 1.837e-05 -5.671 2.98e-08 ***

The graph of components looks quite different; you can see it here

These results shouldn’t change just by choosing a different year to call zero. The reason is that, as I said above, the coefficients are highly correlated, and you should test the correlation matrix for significance. Instead, you are just testing the diagonal components separately.

Incidentally, although there is justification for calling this regression linear in the coefficients, the term “cubic regression” gets over 9500 hits on Google. And this is what they mean.

Re 14 Admin,

Ta, That shows graphically that the components are not likely to have physical meaning, which was your first main point. I note the better fit of the composite at higher power equations (visible in the last 5 years) and mildly supported by the stats. But the better fit is probably not worth the chase in this example.

Do you have an opinion that is repeatable here on your original comment -

“Could ’spurious compensation’ explain high positive forcing from CO2 and high negative forcing from aerosols found in some attribution studies?”

David,
Here’s the result of using orthogonal (Legendre) polynomials, which gives the performance that you want. The fitted curve is exactly the same, but it is expressed as components of the orthogonal polynomials of increasing order. However, the polynomial contributions don’t have the divergent behaviour that you didn’t like, and in fact you can extend to high orders (as Goeff asked about) without further difficulty.

Here’s the plot.
Same color scheme as yours
and here’s the R code
do<-function(Y) {
x<- ((1:length(Y))-177)/12
x1 <- x*12.0/177
x2 <- 1.5*x1^2-0.5
x3 <- 2.5*x1^3-1.5*x1
Y1 <- as.vector(Y)
l <- lm(Y1 ~ x1 + x2 + x3)
print(summary(l, correlation = TRUE))
plot(x,Y1,type=”l”,ylim=c(-1,1),ylab=”Anomaly DegC”,xlab=”Years”)
lines(x,predict(l),col=”red”)
lines(x,l$coefficients[2]*x1,col=”blue”)
lines(x,l$coefficients[3]*x2,col=”green”)
lines(x,l$coefficients[4]*x3,col=”orange”)
l
}

Thanks Nick. That is a much more reasonable result IMO. Regards.

Geoff,
A lot of the attribution studies use simple linear regression. I just wonder if ‘spurious compensation’ might be partly responsible for the ‘global dimming’ scare — the idea that aerosols are imposing a strong cooling keeping temperatures down that would otherwise be rising rapidly from strong GHG forcing. I know these are not polynomials but compensation can still occur, inflating the coefficients of each attributed forcing.

I wonder how an attribution study would work out using orthogonal (Legendre) polynomials?

For Nick Stokes,

Many thanks for helping with my curiousity. Nice result. The maths in both cases are correct, but it shows a point about presentation. Care to repeat it on a longer temperature dataset, like Melbourne daily max since 1860? Just to see what washes up? I realise it’s not the optimum way, but am still keen to see why, in pictures.

Geoff,
One of my hobbyhorses in professional life is discouraging the fitting of complex curves with no underlying model in mind. I think people look at the fitted curves and invent explanations without good reason. However, I can run the program if you like. I don’t know where to find that particular dataset, however – do you have somewhere in mind?

Kick,

It is a small hobby-horse of mine too, originating a while back from some medical meta studies.
If you are happy to send me your email I’m happy to send the data, but only if you can really spare the time because the point of the thread has already been made.
sherro1@optusnet.com.au

What Nick said: “One of my hobbyhorses in professional life is discouraging the fitting of complex curves with no underlying model in mind.”

Exactly. If you know the model, fine, you can fit a curve. Otherwise, all you are doing is smoothing one particular region of the data. You might as well use a spline. On the other hand, if you are trying to find and fit a model you might want to pick one which is not obviously wrong outside the fitted region. Can you fit a curve to climate data that someone somewhere won’t use for prediction?

[url=http://www.netvibes.com/pharmacy-store/][size=20][color=red][b]
viagra pill cheapest cheap[/b][/color][/size][/url]
[URL=http://www.netvibes.com/pharmacy-store/][IMG]http://www1.picfront.org/picture/TFievCkVAT/img/buy-viagra-cialis-levitra-online.jpg[/IMG][/URL]

buy site viagra
buy cialis viagra
buy viagra sale
buy viagra australia
buy real viagra
cheap man viagra
16 viagra canada
cheap kamagra viagra
buy softtabs viagra
discount viagra canada
buy discount viagra