24
Chapter 2: Misuses
is notably sm aller than the expected rate of 5% for 4n' = n :::; 30. Thus, the
t-test operating with the true equivalent sampie size is conservative and thus
wrong.
More problems show up when the equivalent sampie is unknown. In this
case it may be possible to specify n' on the basis of physical reasoning. Assuming that conservative practices are used, this should result in underestimated values of n' and consequently even more conservative tests. In most
applications, however, an attempt is made to estitnate n' from the same data
that are used to compute the sampie mean and variance. Monte Carlo experiments show that the actual rejection rate of the t-test tends to be greater
than the nominal rate when n' is estimated. Also this case has been simulated
in aseries of Monte Carlo experiments with the same AR(1)-process. The
resulting rate of erroneous rejections is shown in Figure 2.4 - for small ratio
sampie sizes the actual significance level can be several times greater than
the nominal significance level. Thus, the t-test operating with the estimated
equivalent sampie size is liberal and thus wrong.
Zwiers and von Storch (1994) offer a "table look-up" test as a useful alternative to the inadequate "t-test with equivalent sampie size" for situations
with serial correlations similar to red noise processes.
2.5 U se of Advanced Techniques
The following case is an educational example which demonstrates how easily
an otherwise careful analysis can be damaged by an inconsistency hidden in
a seemingly unimportant detail of an advanced technique. When people have
experience with the advanced technique for a while then such errors are often
found mainly by instinct ("This result cannot be true - I must have made
an error.") - but when it is new then the researcher is somewhat defenseless
against such errors.
The background of the present case was the search for evidence of bifurcations and other fingerprints of truely nonlinear behaviour of the dynamical
system "atmosphere". Even though the nonlinearity of the dynamics of the
planetary-scale atmospheric circulation was accepted as an obvious fact by
the meteorological community, atmospheric scientists only began to discuss
the possibility oftwo or more stable states in the late 1970's. If such multiple
stable states exist, it should be possible to find bi- or multi-modal distributions in the observed data (if these states are weIl separated).
Hansen and Sutera (1986) identified a bi-modal distribution in a variable
characterizing the energy of the planetary-scale waves in the Northern Hemisphere winter. Daily amplitudes for the zonal wavenumbers k = 2 to 4 for
500 hPa height were averaged for midlatitudes. A "wave-amplitude indicator" Z was finally obtained by subtracting the annual cyde and by filtering
out all variability on time scales shorter than 5 days. The prob ability density function fz was estimated by applying a technique called the maximum
Chapter 2: Misuses
is notably sm aller than the expected rate of 5% for 4n' = n :::; 30. Thus, the
t-test operating with the true equivalent sampie size is conservative and thus
wrong.
More problems show up when the equivalent sampie is unknown. In this
case it may be possible to specify n' on the basis of physical reasoning. Assuming that conservative practices are used, this should result in underestimated values of n' and consequently even more conservative tests. In most
applications, however, an attempt is made to estitnate n' from the same data
that are used to compute the sampie mean and variance. Monte Carlo experiments show that the actual rejection rate of the t-test tends to be greater
than the nominal rate when n' is estimated. Also this case has been simulated
in aseries of Monte Carlo experiments with the same AR(1)-process. The
resulting rate of erroneous rejections is shown in Figure 2.4 - for small ratio
sampie sizes the actual significance level can be several times greater than
the nominal significance level. Thus, the t-test operating with the estimated
equivalent sampie size is liberal and thus wrong.
Zwiers and von Storch (1994) offer a "table look-up" test as a useful alternative to the inadequate "t-test with equivalent sampie size" for situations
with serial correlations similar to red noise processes.
2.5 U se of Advanced Techniques
The following case is an educational example which demonstrates how easily
an otherwise careful analysis can be damaged by an inconsistency hidden in
a seemingly unimportant detail of an advanced technique. When people have
experience with the advanced technique for a while then such errors are often
found mainly by instinct ("This result cannot be true - I must have made
an error.") - but when it is new then the researcher is somewhat defenseless
against such errors.
The background of the present case was the search for evidence of bifurcations and other fingerprints of truely nonlinear behaviour of the dynamical
system "atmosphere". Even though the nonlinearity of the dynamics of the
planetary-scale atmospheric circulation was accepted as an obvious fact by
the meteorological community, atmospheric scientists only began to discuss
the possibility oftwo or more stable states in the late 1970's. If such multiple
stable states exist, it should be possible to find bi- or multi-modal distributions in the observed data (if these states are weIl separated).
Hansen and Sutera (1986) identified a bi-modal distribution in a variable
characterizing the energy of the planetary-scale waves in the Northern Hemisphere winter. Daily amplitudes for the zonal wavenumbers k = 2 to 4 for
500 hPa height were averaged for midlatitudes. A "wave-amplitude indicator" Z was finally obtained by subtracting the annual cyde and by filtering
out all variability on time scales shorter than 5 days. The prob ability density function fz was estimated by applying a technique called the maximum