On Zen and the Art of Climate Analysis
by Greg Goodman
The author has a graduate degree in applied physics, professional experience in spectroscopy, electronics and software engineering, including 3-D computer modelling of scattering of e-m radiation in the Earth’s atmosphere.
In trying to understand changes in climate it would be logical to look at the rate of change directly rather than trying to guess at and its causes by looking at the various time series of temperature data.
This is also important since most of what climate science refers to as “forcings” are power terms measured in W/m2 . Temperature is a measure of energy, so it is the rate of change of temperature that reflects power. If there is a change in radiative ‘forcing’ such as solar or CO2, the response will be instantaneous in rate of change, however the subsequent change in temperature will take time to accumulate before it becomes evident, by which time some other factors have probably already obscured or confused the signal.
Most of what follows will be looking at rate of change of temperature : dT/dt, in particular sea surface temperature as contained in the International Comprehensive Ocean-Atmosphere Data Set ( ICOADS )
It is important in attempting to untangle the various forces acting upon climate to identify the various effects and their causes and their long-term interactions. It is not sufficient to dismiss everything that happens in the climate system as “internal variation” and assume that it all necessarily averages out. There is plenty of evidence that many of these effects are not “internal” but driven by forces from outside. El Nino and La Nina are not symmetrically opposite and equal phases of the same process. Each operates in a very different way. The assumption that they all average out over any particular time-scale is spurious.
The major oceanic basins have largely independent masses of water, at least in the upper levels. due to the large circulating currents referred to as gyres, driven by the Coriolis force . For this reason the major oceans are examined separately here. They are further broken down into their tropical and extra-tropical regions, since climate patterns are markedly different in the tropics. This helps in identifying underlying patterns that may be lost by the common, simplistic notion of global mean temperatures.
A useful way to examine whether there are repetitive patterns in a data series is by examining the autocorrelation function. This is generated by overylaying the data on itself with a progressively increasing offset in time. The correlation coefficient being calculated at each step. This method does not assume anything about the nature or form of any repetition and so is not imposing or assuming any structure on the data.
One way to extract the underlying frequencies is to look at the power density spectrum (PDS). This shows the strength of each frequency or band of frequencies. This is usually done on the square of the power. As already noted, dT/dt is a power term.
The PDS can be derived by taking the Fourier transform of the autocorrelation function. Doing so reveals a large number of oscillations are present in all basins, many of them common to several basins.
One problem with using the rate of change is that it amplifies shorter signals by 1/f and attenuates longer ones . The higher frequency terms, which in this context are the ‘noise’ of weather on the climate signal, get bigger. To prevent this h.f. swamping the signal of interest, a 12 month gaussian filter is used to attenuate the annual and shorter variations.
Original ICOADS SST was chosen for this study since there are notable differences in the autocorrelation and the power spectra of the re-gridded, re-processed and “corrected” Hadley SST datasets.
Examination of the literature detailing the calculation of the ‘climatology’ used in preparing HadSST3 (Brohan 2006 etc.) shows that crude running-mean filters are applied across adjacent grid cells to smooth the data and supposedly reduce noise. However, this implies both spatial and temporal phase shifts as well as introducing the frequency distortions inherent in using running means as a filter. This problem is aggravated since the process is repeated in a loop until the climatology “converges” to a stable result.
There is no evidence in the associated papers that any assessment has ever been made as to the effect that this kind of precessing has on the frequency characteristics of the data but they are substantial. It would seem incumbent on the authors of such work to demonstrate that this heavy processing is improving the data rather than distorting and degrading it. In the absence of such a assessment, it was considered more appropriate to work with the unprocessed data from ICOADS which , while not being without it’s own problems, probably better represents the frequency structure of the historical record .
This brief overview looks at groups of basins to examine the presence and period of common cyclic features.
The following is an example for the North Atlantic and South Pacific oceans.
Figure 1. Autocorrelation function of N. Atlantic , S. Pacific
It is clear that there are strong repetitive patterns in the data. In fact, it is surprising how similar these two basins are considering they are in opposite hemispheres and physically separate.
Another feature that is instantly obvious to anyone with experience in signal processing is that there is more than one major frequency present. The recurrent dips in amplitude are classic signs of an interference pattern between at least two, closely related frequencies.
The significance of the peaks and troughs in correlation depends upon the number of data used. So as the offset increases and there are less points that overlap and a given level of correlation is less significant (more possibly just chance). This is accounted for in the line showing 95% significance probability. The patterns are clearly of a significant level and not due to random variations.
Earth’s climate is a very complex system of interactions so it is often more appropriate to think in terms of pseudo-cycles than pure simple harmonic functions, though the underlying physical mechanisms can often produce basic forces and reactions that would lead to harmonic oscillations in a simpler context.
The power spectrum of the Nino 1 region, off the coast of Peru, is one of the least complicated. Notable, is the circa 13 year cycle and its second harmonic around 6.5 years. With the three nearby peaks around 5 years and the longer terms this will appear to be chaotic and unpredictable when viewed as a time series of temperature. But it can be seen to be highly structured once correctly analysed.
Figure 2. Power spectrum of El Nino 1 region.
It can be seen that many of the peaks are harmonic pairs and that they correspond to the various frequencies that Bart’s artful analysis of sun-spot number predicted:
He calculated the following periods: 5, 5.2, 5.4, 5.6, 5.9, 9.3, 10, 10.8, 11.8, 13, 20, 23.6 , 65.6 and 131 years in sun-spot number.
In addition to Bart’s periods, which were solely related to solar cycle itself, not it’s effects on terrestrial climate, there is another reason to expect several higher harmonics to be present in SST data.
When a radiative driver of climate, such as solar or CO2, varies there will an additional radiative forcing on the surface. This will instantly produce a rate of change of temperature. Once a higher (or lower) temperature is attained there will be a difference in emitted radiation and evaporation, two processes that depend upon temperature and provide a negative feedback. The temperature change will be the mathematical integral of the radiative forcing over time and will have a phase delay. In the case of a cyclic change, cosine will integrate to sine. The new radiative effect will, in turn, produce a change in temperature 90 degrees out of phase with the original, and so the loop repeats.
There is a trigonometrical identity that sin(x).cos(x)=cos(2x) , so mixing such a rate of change with its integral in this way will produce a signal of twice the frequency: the second harmonic. and so on. An equal mix will produce just the harmonic, a more likely unequal mix will result in both fundamental and harmonic being present.
So in the case of SST we should expect to find a series of pure harmonics being produced quite naturally.
The very weak peaks 10.26 and 11.2 (not annotated explicitly on the graph for the sake of clarity, but visible) match the peaks at 5.15, 5.7 and a tiny 2.86 peak .
These harmonics are indeed found to be very common.
It can be seen that the short harmonics lose power in favour of the longer periods as the Nino/Nina patterns progress from the peruvian Nino 1 to the central Nino 3.4 region.
Figure 2b. Power spectra of Nino 1 and Nino 3.4 regions
Figure 3. Power spectra ensemble for basins sharing 9 and 13 year peaks.
It can be seen that the 9 year cycle is clearly defined and stable. The 13 year shows greater variation between basins and is broader, showing more variation in the length of the pseudo cycle. This is typical of the pseudo cycles in solar SSN data.
Figure 4. Collection of power spectra for basins sharing 9 year peaks.
It is noted that the 9 year signal is prominent right across North and South, Atlantic and Pacific oceans in both tropical and extra-tropical zones.
Figure 5. Nicola Scafetta’s graph  , derived from NASA/JPL ephemeris, demonstrating a circa 9 year periodicity in variations of the Earth’s orbital distance from the sun, caused by the presence of the moon. The results of the MEM spectral analysis shows a period of 9.1 +/- 0.1 that corresponds closely to periodic fluctuations seen here in SST. These are clearly distinct from usually noted 8.85 and 9.3 year lunar tidal periods.
The presence of a strong 9 year cycle prevents a simple identification of a link between solar activity, as witnessed by sun-spot number ( SSN ) and surface temperatures. Figure 6 below shows a comparative graph. It can be seen that at the end of the 19th century the two cycles were in phase and reasonably in step with SST. By 1920 the individual peaks are seen interspersed as the cycles are totally out of phase. There is a phase crisis in the relationship with SST. By 1960 they are back in phase and, by the end of the millennium, can be seen to be once again diverging.
This lack of any clear correspondence has often been cited as proof that there is no discernible link between global temperatures and SSN.
Figure 6 comparing rate of change of SST to sun-spot area.
Little published work seems to exist on this apparently strong lunar connection to climate. Prof.Harald Yndestad  has published several papers on the subject that also provide strong evidence, however they are very restricted in geographical scope and draw conclusions that are limited to suggesting a regional effect.
What is shown here is a much more significant, global effect. The presence of this strong 9 year cycle will confound attempts to detect the solar signal unless it is recognised. When the two are in phase (working together) the lunar effect will give an exaggerated impression of the scale of the solar signal and when they are out of phase the direct relationship between SSN and temperatures breaks down, leading many to conclude that any such linkage is erroneous or a matter of wishful thinking by less objective observers.
Such long term tidal or inertial effects can shift massive amounts of water and hence energy in and out of the tropics and polar regions. Complex interactions of these cycles with others, such as the variations in solar influence, create external inputs to the climate system, with periods of decadal and centennial length. It is essential to recognise and quantify these effects rather than making naive and unwarranted assumptions that any long term changes in climate are due to one simplistic cause such as the effects of trace gas like CO2.
It is totally unfounded to suggest that these effects will simply average out over the time-scale of the current surface temperature record without detecting and characterising their form, duration and interaction. Even more so to ignore their importance in the much quoted “latter half of the 20th century”.
Failure to recognise the importance of these not-so-internal cycles in climate variation likely accounts for the thorough failure of attempts to model and predict climate so far. Efforts which have now so obviously mislead our expectations over the first decades in which their predictions have been used to promote massive policy changes. Somewhat belated recognition that there are fundamental problems in current models has resulted in the recent reassessment of the projections made by Met. Office Hadley, from one of continued and alarming rise to one of five more years of non rising global temperatures following the last 16 years of no significant change.
Further work is needed in identifying and explaining these variations to determine the role they have played in recent changes in surface temperature before attempts are made to predict future variations.
Notes on file names used in graphs:
icoads indicates ICOADS v2.5 SST data
autocorr refers to auto-correlation function
g12m indicates 12m 3-sigma gaussian filter
ddt indicates time derivative dT/dt
s1200m indicates a sample of 1200 monthly temperatures
‘using 3:4’ is simply the data columns being plotted.
Regional ocean areas used:
the following regular shaped regions were selected to represent the basins used:
SST data were obtained from KNMI archive. Early years with many gaps were removed and sparse gaps were filled by averaging same month from previous and following year. This is crude but for the small number of cases will not have noticeable effect on the spectra.
Daily sun-spot area was provided by: http://solarscience.msfc.nasa.gov
(set x=y in equation 6.4)
“Empirical evidence for a celestial origin of the climate oscillations and its implications”
N. Scafetta / Journal of Atmospheric and Solar-TerrestrialPhysics 72 (2010) 951–970
Pay-walled paper but many clear and explicit graphs available:
Many thanks to Tim Channon for providing the software used for Fourier analysis.
To the R project which was used to derive the autocorrelation series
To Gnuplot team for their amazingly powerful plotting software
To those sifting and maintaining the massive ICOADS dataset and the million seamen who have braved the elements through centuries of storms and wars to throw their buckets overboard
Finally to Steve Mosher for inspiring me to look into the question of celestial causes for climate change.
Useful and highly interesting paper by Ian Wilson showing the planetary origins of many of these peaks.
Figure A1. showing comparison of HadSST3 and ICOADS power density spectrum for extra-tropical North Pacific. Spectra are similar, but with significant differences between 7 and 9 years.