Section 2.5: Epilogue
25
penalty technique to 16 winters of daily data. The resulting fz had two maxima separated by a minor minimum. This bimodality was taken as proof
of the existence of two stable states of the atmospheric general circulation:
A "zonal regime", with Z < 0, exhibiting small amplitudes of the planetary
waves and a "wavy regime", with Z > 0, with amplified planetary-scale zonal
disturbances.
Hansen and Sutera performed a "Monte Carlo" experiment to evaluate the
likelihood of fitting abimodal distribution to the data with the maximum
penalty technique even if the generating distribution is unimodal. The authors concluded that this likelihood is smalI. On the basis of this statistical
check, the found bimodality was taken for granted by many scientists for
almost a decade.
When I read the paper, I had never heard ab out the "maximum penalty
method" but had no doubts that everything would have been done properly
in the analysis. The importance of the quest ion prompted other scientists to
perform the same analysis to further refine and verify the results. Nitsche et
al. (1994) reanalysed step-by-step the same data set which had been used in
the original analysis and came to the conclusion that the purportedly small
prob ability for amisfit was large. The error in the original anlysis was not
at all obvious. Only by carefully scrutinizing the pitfalls of the maximum
penalty technique did Nitsche and coworkers find the inconsistency between
the Monte Carlo experiments and the analysis of the observational data.
Nitsche et al. reproduced the original estimation, but showed that something like 150 years of daily data would be required to exclude with sufficient
certainty the possibility that the underlying distribution would be unimodal.
What this boils down to is, that the null hypothesis according to which the
distribution would be unimodal, is not rejected by the available data - and
the published test was wrong. However, since the failure to reject the null
hypothesis does not imply the acceptance of the null hypothesis (but merely
the lack of enough evidence to reject it), the present situation is that the
(alternative) hypothesis "The sampie distribution does not originate from a
unimodal distribution" is not falsified but still open for discussion.
I have learned the following rule to be useful when dealing with advanced
methods: Such methods are often needed to find a signal in a vast noisy phase
space, i.e., the needle in the haystack - but after having the needle in our hand,
we should be able to identify the needle as a needle by simply looking at it. 7
Whenever you are unable to do so there is a good chance thai something is
rotten in the analysis.
7See again Wallace's and Gutzler's study who identified their teleconnection patterns
first by examining correlation maps - and then by simple weighted means of few grid point
values - see Section 12.1.
25
penalty technique to 16 winters of daily data. The resulting fz had two maxima separated by a minor minimum. This bimodality was taken as proof
of the existence of two stable states of the atmospheric general circulation:
A "zonal regime", with Z < 0, exhibiting small amplitudes of the planetary
waves and a "wavy regime", with Z > 0, with amplified planetary-scale zonal
disturbances.
Hansen and Sutera performed a "Monte Carlo" experiment to evaluate the
likelihood of fitting abimodal distribution to the data with the maximum
penalty technique even if the generating distribution is unimodal. The authors concluded that this likelihood is smalI. On the basis of this statistical
check, the found bimodality was taken for granted by many scientists for
almost a decade.
When I read the paper, I had never heard ab out the "maximum penalty
method" but had no doubts that everything would have been done properly
in the analysis. The importance of the quest ion prompted other scientists to
perform the same analysis to further refine and verify the results. Nitsche et
al. (1994) reanalysed step-by-step the same data set which had been used in
the original analysis and came to the conclusion that the purportedly small
prob ability for amisfit was large. The error in the original anlysis was not
at all obvious. Only by carefully scrutinizing the pitfalls of the maximum
penalty technique did Nitsche and coworkers find the inconsistency between
the Monte Carlo experiments and the analysis of the observational data.
Nitsche et al. reproduced the original estimation, but showed that something like 150 years of daily data would be required to exclude with sufficient
certainty the possibility that the underlying distribution would be unimodal.
What this boils down to is, that the null hypothesis according to which the
distribution would be unimodal, is not rejected by the available data - and
the published test was wrong. However, since the failure to reject the null
hypothesis does not imply the acceptance of the null hypothesis (but merely
the lack of enough evidence to reject it), the present situation is that the
(alternative) hypothesis "The sampie distribution does not originate from a
unimodal distribution" is not falsified but still open for discussion.
I have learned the following rule to be useful when dealing with advanced
methods: Such methods are often needed to find a signal in a vast noisy phase
space, i.e., the needle in the haystack - but after having the needle in our hand,
we should be able to identify the needle as a needle by simply looking at it. 7
Whenever you are unable to do so there is a good chance thai something is
rotten in the analysis.
7See again Wallace's and Gutzler's study who identified their teleconnection patterns
first by examining correlation maps - and then by simple weighted means of few grid point
values - see Section 12.1.