Previously, I posted on the multiple regression method – in particular, the method employed in Foster and Rahmstorf (2011) – and how, when attempting to decompose the temperature evolution of my simple energy balance model into the various components (signal, ENSO, solar, and volcanic), this method encountered two large issues:
1) It did not adequately identify the longer term effect of the volcanic recovery on temperature trends, and
2) It largely overestimated the solar influence.
If you recall, I tested two scenarios in that original post. The first scenario was a linearly increasing underlying signal. The second scenario was a combination of a linearly increasing signal and an underlying low-frequency oscillation, resulting in a flattening of recent temperatures (one that was not caused by the combination of ENSO, volcanic, and solar influences). The goal was to see whether this multiple regression method could identify the flattening if it existed.
Thanks to Kevin C, who suggested and implemented a few improvements to this F&R method, noting them in the comments of that post: “…tie the volcanoes and solar together as forcings and fit a single exponential response term instead of a delay." This would allow a tail for the recovery from volcanic eruptions well beyond the removal of that actual stratospheric aerosols, and would not allow an over-fitting of the solar influence. After implementing this newer method, I would say that it is a large improvement (at least in diagnosing my simple EBM components) in the first scenario of a linearly increasing trend:
Unfortunately, due to the underlying assumption implicit in this method of a linear trend, it still has trouble identifying the recent pause present in scenario 2:
To see where exactly it is going wrong in scenario 2 vs. scenario 1, we can again look at the individual components:
As should be clear, the improvements suggested by Kevin C generally improve performance across the board. Unfortunately, in the 2nd scenario with the flattening, the multiple regression method still tries to compensate for the flattening by decreasing the diagnosed influence of volcanic recovery, therefore leading to a misdiagnosis.
Dikran Marsupial noted in the comments of that last post that “there are no free lunches.” Perhaps this helps drive the point home that assuming an underlying linear trend will lead to this misdiagnosis if the increase is not linear. I hope to investigate further the actual influence of solar + volcanic activity on recent temperatures using some GCM runs.
Troy's Scratchpad 
And I agree with Troy’s analysis. In fact I now see at least two problems with my method:
1. The assumption of a linear trend. The hope here is that the model is parsimonius enough that it can’t significantly compensate for non-linearity by distorting the other terms. What Troy has shown is that this is not the case. My version performs better because there are two fewer parameters (and possibly because the model is more realistic), but the problem is still present.
I actually experimented with a quadratic trend term, and a trend discontinuity at 1997, but rejected both on the basis of AIC. Those would probably have helped with scenario 2, but only because they happen to match the scenario – the underlying problem would remain and would be picked up by other scenarios.
2. Failure to account for hidden parameters in the uncertainties. There is a parameter in my calculation which does not show up in the regression – the exponential lag term (F&R has 2 delay terms). There is significant uncertainty in the calculation of this lag, and as we now see the lag can play a significant role in shaping the results. My stats aren’t strong enough to be sure of how to handle this. Non-linear regression is presumably a possibility, but I know that in the case of the 2-box model it fails to find the minima – I have to use a grid search to find the best lags.
My best guess at the right approach (and maybe a real statistician can help here) is to calculate the results for every lag, and the combine the distributions of the parameters at each lag, weighted according to the likelihood of the model for that lag. There’s probably not much mileage in fixing this at this point, but I’m interested to know how I should have done it.
My conclusion from this is that F&R and myself have dramatically underestimated the uncertainties in our attempts at attribution of the recent warming trend. It remains a very interesting problem, and I’m looking forward to Troy’s (and anyone else’s) future results.
Comment by Kevin C — February 21, 2013 @ 3:17 am
For a method to be able to diagnose non-linear behavour, the analysis needs to include the option for non-linear behaviour so that the analysis has the option to chose it. As I said, I don’t think this was the purpose of F&R, however diagnosis of non-linear behaviour is an interesting topic. I would however start by looking at change point detection, however then you run into the opposite proble, namely detecting a non-linear behaviour when it doesn’t actually exist (e.g. http://tamino.wordpress.com/2012/01/10/step-2/).
This is why usual statistical practice is to only go with a more complex hypothesis if you can first rule out the possibility that a more basic hypothesis will do (Occam’s razor). This is what we do in fitting a linear trend in the first place – we only claim there is warming ON PURELY STATISTICAL GROUNDS if we can first show that the obsevations are inconsistent with the underlying temperature being constant. Likewise we should only assert the existence of a step change in temperatures or in the rate of warming if we can first show that the observations are inconsistent with warming at a constant rate (I suspect the purpose of F&R is to show that this currently is not possible). This self-skeptical approach to statistical evidence (while flawed) has served science rather well for the last century.
So the key question should be, “is there statistically significant support from the observations for the existence of non-linear behaviour?”
Comment by dikranmarsupial — February 21, 2013 @ 3:54 am
DM, I would have to disagree with this since F&R state explicitly in their conclusion that “There is no indication of any slowdown or acceleration of global warming, beyond the variability induced by these known natural factors.” Of course F&R were not attempting to diagnose a non-linear signal, but wouldn’t you agree that “slowdown” necessarily implies non-linearity? Since, as Troy has shown, the F&R method is not capable of fitting to a “slowdown” then how can they justify such a conclusion?
Comment by Layman Lurker — February 21, 2013 @ 7:28 am
The conclusion is reasonable as far as I can see. They are saying merely that the apparent slowdown is completely explanable as an artefact of known variability, which their analysis does demonstrate. This is essentially standard practice in statistics.
Consider this example, if you suspect a coin is biased. The normal (frequentist) statistical practice would be to perform an hypothesis test. Let r be the probability of getting a head, then we would normally start by stating the null and alternative hypotheses:
H0 – the coin is unbiased, i.e. r = 0.5
H1 – the coin is biased, i.e. r != 0.5
We then compute the p-value, which is the probability of observing a test statistic at least as extreme, assuming that H0 is true. If the p-value is less than some threshold (often 0.05) we say “we reject the null hypothesis”; if it is greater than the threshold, “we fail to reject the null hypothesis”. To make it easy, lets assume that the test statistic is the number of heads observed in n flips, and we observe 4 heads in 4 flips, which gives us a p-value of 0.0625, so we “fail to reject the null hypothesis”.
The important thing to note here, is that the result of the test does not depend in any way on H1, only on H0. It does not matter whether H0 is actually correct or not. The underlying principle of the test is one of self-skepticism, we only proceed with our research hypothesis H1 if and only if we are able to reject the null hypothesis (i.e. the opposite of what we would like to be true).
Now in the case of testing for a step change in the rate of warming, we start by formulating our alternative and null hypotheses:
H1 – Once known forms of variability are accounted for, there has been a change in the underlying rate of warming (i.e. the rate of warming is non-linear)
H0 – Once known forms of variability are accounted for, the underlying rate of warming is linear
and then see how unlikely the observations would be, assuming that H0 were true. This is essentially the point of the F&R analysis, they have shown that the observations are consistent with H0, and hence it can’t be rejected.
The onus is on those who wish to assert that there has been some meaningful non-linearity to demonstrated that they are able to reject an appropriate (non-straw-man) null hypothesis, as it is they that are arguing for a change, or equivalently for a more complicated hypothesis.
Note F&R don’t claim that their analysis demonstrated that there has been no step change or non-linearity, merely that there is no indication of one (beyond known sources of variability). Unfortunately frequentist statistics are rather subtle, and these nuances in interpretation are of vital importance.
Comment by dikranmarsupial — February 21, 2013 @ 7:53 am
Following dikran:
F&R cannot possibly *exclude* the 60y oscillation – simply because over 30y, the difference between 60y cycle + low linear trend, and high linear trend, is smaller than the noise.
What they show is that you don’t *need* an oscillation.
It’s a response to the claims that the recent slowdown in warming must imply a slowdown in the underlying signal. Well, no it doesn’t.
In your model the oscillation is indeed needed to match temperatures. But IIUC that’s because of the peculiar volcanic responses of your model, that “make room” for the oscillation. Is it for real? I’d like to see other modelers weigh on that.
Maybe Gavin will touch a word of this in his promised “part III” on sensitivity.
Comment by toto — February 23, 2013 @ 7:46 am
Toto,
You are in luck, as I have been working on a post that examines the natural-only GCM runs (7 models, 39 runs in all), to see if they indeed suggest a longer-tail for the volcanic response, similar to my model. I have just put it up, and if you check out https://troyca.wordpress.com/2013/02/23/could-the-multiple-regression-approach-detect-a-recent-pause-in-global-warming-part-3/ , I think you will find the AOGCMs generally tell a similar story.
I suppose I agree that the recent slowdown in warming does not *necessarily* require “a slowdown in the underlying signal.” And I also don’t think it is worth getting into some semantic battle over what FR11 is actually trying to show (and whether that is “right” or “wrong”). Rather, in the interest of moving forward, I would hope the three posts on this topic have highlighted the following:
1) This multiple regression method is unable to detect a pause in recent warming, even if it is present in the signal. It reconstructs the pause out of existence by underestimating the magnitude and length of the volcanic response (and in the case of FR11 specifically, will overestimate the solar response).
2) The magnitude and length of the volcanic response is much lower in the FR11 diagnosis than from physics-based models (whether they be GCMs or simple energy balance), and the solar response is greater.
3) Given that #2 is exactly what we would expect if there *was* a recent pause and the multiple regression method was forced to compensate, I would say this is a strong reason to consider some meaningful non-linearity.
Comment by troyca — February 23, 2013 @ 12:41 pm
[…] Part 2 […]
Pingback by Could the multiple regression approach detect a recent pause in global warming? Part 3. | Troy's Scratchpad — February 23, 2013 @ 12:22 pm
[…] a dream is all it is. When you do the analysis correctly (like here and here and here) you see what happens when you wake up to reality. Clearly, Rose’s article stating “The Mail on […]
Pingback by Global Warming Reality or a Myth. Excellent read below. | envirofrogblog — March 18, 2013 @ 9:57 am
[…] a dream is all it is. When you do the analysis correctly (like here and here and here ) you see what happens when you wake up to reality. Clearly, Rose’s article stating “The Mail on […]
Pingback by Global warming denial: Debunking misleading climate change claims by David Rose. | Interesting Things — March 29, 2013 @ 7:16 am
[…] has been a while since I posted the first three parts of a series on whether using multiple linear regressions to remove the solar, volcanic, and […]
Pingback by Another “reconstruction” of underlying temperatures from 1979-2012 | Troy's Scratchpad — May 21, 2013 @ 8:26 am
[…] 21/02/2013: Troy Masters is doing some interesting analysis on the methods employed here and by Foster and Rahmstorf. On the basis of his results and my latest […]
Pingback by Skeptical Science Folly: Video Based On Flawed Rahmstorf & Foster Paper Disappears! — June 13, 2013 @ 6:03 am