Section 2.2: Mandatory Testing and the Mexican Hat
13
2.2 Mandatory Testing and the Mexican Hat
In the desert at the border of Utah and Arizona there is a famous combination
of vertically aligned stones named the "Mexican Hat" which looks like a
human with a Mexican hat. It is a random product of nature and not manmade ... really? Can we test the null hypothesis "The M exican Hat is 0/
natural origin"? To do so we need a test statistic for a pile of stones and a
prob ability distribution for this test statistic under the null hypothesis. Let's
take
t( ) = {1 if p for~s a Mexican Hat
p
0 otherwlse
(2.1)
for any pile ofstones p. How do we get a prob ability distribution oft(p) for all
piles ofstones p not affected by man? - We walk through the desert, examine
a large number, say n = 10 6 , of piles of stones, and count the frequency of
t(p) = 0 and of t(p) = 1. Now, the Mexican Hat is famous for good reasons
- there is only one p with t(p) = 1, namely the Mexican Hat itself. The
other n - 1 = 10 6 - 1 sampIes go with t(p) = O. Therefore the prob ability
distribution for p not affected by man is
{
10-6
for k = 1
prob (t(p) = k) = 1 _ 10-6 for k = 0
(2.2)
After these preparations everything is ready for the final test. We reject the
null nypothesis with a risk of 10- 6 if t(Mexican hat) = 1. This condition is
fulfilled and we may conclude: The Mexican Hat is not 0/ natural origin hut
man-made.
Obviously, this argument is pretty absurd - but where is the logical error?
The fundamental error is that the null hypothesis is not independent 0/ the
data which are used to conduct the test. We know a-priori that the Mexican
Hat is a rare event, therefore the impossibility of finding such a combination
of stones cannot be used as evidence against its natural origin. The same
trick can of course be used to "prove" that any rare event is "non-natural",
be it a heat wave or a particularly violent storm - the prob ability of observing
a rare event is small.
One might argue that no serious scientist would fall into this trap. However,
they do. The hypothesis of a connection between solar activity and the
statistics of climate on Earth is old and has been debated heatedly over
many decades. The debate had faded away in the last few decades - and
has been refuelled by a remarkable observation by K. Labitzke. She studied
the relationship between the solar activity and the stratospheric temperature
at the North Pole. There was no obvious relationship - but she saw that
during years in which the Quasibiennial Oscillation (QBO) was in its West
Phase, there was an excellent positive correlation between solar activity and
North Pole temperature whereas during years with the QBO in its East Phase
13
2.2 Mandatory Testing and the Mexican Hat
In the desert at the border of Utah and Arizona there is a famous combination
of vertically aligned stones named the "Mexican Hat" which looks like a
human with a Mexican hat. It is a random product of nature and not manmade ... really? Can we test the null hypothesis "The M exican Hat is 0/
natural origin"? To do so we need a test statistic for a pile of stones and a
prob ability distribution for this test statistic under the null hypothesis. Let's
take
t( ) = {1 if p for~s a Mexican Hat
p
0 otherwlse
(2.1)
for any pile ofstones p. How do we get a prob ability distribution oft(p) for all
piles ofstones p not affected by man? - We walk through the desert, examine
a large number, say n = 10 6 , of piles of stones, and count the frequency of
t(p) = 0 and of t(p) = 1. Now, the Mexican Hat is famous for good reasons
- there is only one p with t(p) = 1, namely the Mexican Hat itself. The
other n - 1 = 10 6 - 1 sampIes go with t(p) = O. Therefore the prob ability
distribution for p not affected by man is
{
10-6
for k = 1
prob (t(p) = k) = 1 _ 10-6 for k = 0
(2.2)
After these preparations everything is ready for the final test. We reject the
null nypothesis with a risk of 10- 6 if t(Mexican hat) = 1. This condition is
fulfilled and we may conclude: The Mexican Hat is not 0/ natural origin hut
man-made.
Obviously, this argument is pretty absurd - but where is the logical error?
The fundamental error is that the null hypothesis is not independent 0/ the
data which are used to conduct the test. We know a-priori that the Mexican
Hat is a rare event, therefore the impossibility of finding such a combination
of stones cannot be used as evidence against its natural origin. The same
trick can of course be used to "prove" that any rare event is "non-natural",
be it a heat wave or a particularly violent storm - the prob ability of observing
a rare event is small.
One might argue that no serious scientist would fall into this trap. However,
they do. The hypothesis of a connection between solar activity and the
statistics of climate on Earth is old and has been debated heatedly over
many decades. The debate had faded away in the last few decades - and
has been refuelled by a remarkable observation by K. Labitzke. She studied
the relationship between the solar activity and the stratospheric temperature
at the North Pole. There was no obvious relationship - but she saw that
during years in which the Quasibiennial Oscillation (QBO) was in its West
Phase, there was an excellent positive correlation between solar activity and
North Pole temperature whereas during years with the QBO in its East Phase