15 Statistical Issues in Particle Physics
675
to statistical issues that arise in discovery-orientated analyses at the LHC. Ref
[74] deals with statistical issues that occur in Particle Physics searches for new
phenomena; as an example, it includes the successful search for the Higgs boson
at the LHC. A more detailed description of the plans for the Higgs search before its
discovery is in ref. [75].
15.7.1 H 0 , or H 0 Versus H 1 ?
In looking for new physics, there are two distinct types of approach. We can compare
our data just with the null hypothesis H 0 , the SM of Particle Physics; alternatively
we can see whether our data are more consistent with H 0 or with an alternative
hypothesis H 1 , some specific manifestation of new physics, such as a particular
form of quark and/or lepton substructure. The former is known as ‘goodness of fit’,
while the term ‘hypothesis testing’ is often reserved for the latter.
Each of these approaches has its own advantage. By not specifying a specific
alternative, 22 the goodness of fit test may be capable of detecting any form of
deviation from the SM. On the other hand, if we are searching for some specific
new effect, a comparison of H 0 and H 1 is likely to be a more sensitive way for that
particular alternative. Also, the ‘hypothesis testing’ approach is less likely to give a
false discovery claim if the assumed form of H 0 has been slightly mis-modelled.
15.7.2 p-Values
In order to quantify the chance of the observed effect being due to an uninteresting
statistical fluctuation, some statistic is chosen for the data. The simplest case would
be the observed number n 0 of interesting events. Then the p-value is calculated,
which is simply the probability that, given the expected background rate b from
known sources, the observed value would fluctuate up to n 0 or larger. In more
complicated examples involving several relevant observables, the data statistic may
be a likelihood ratio L 0 /L 1 for the likelihood of the null hypothesis H 0 compared
with that for a specific alternative H 1 .
To compute the p-value of the observed or of possible data, the distribution f (t)
of the data statistic t under the relevant hypothesis is required. In some cases this
can be obtained analytically, but in more complicated situations, f (t) may require
simulation. For t being −2 ln L 0 /L best , Cowan et al have given useful asymptotic
22 Even a test of the null hypothesis may not be completely independent of ideas about alternatives.
Thus in an event counting experiment, new physics usually results in an increase in rate, unless
we are looking for neutrino oscillations, in which case a decrease would be significant. Also,
sometimes the statistic used for a goodness of fit test of H 0 may be the likelihood ratio for H 0 as
compared with a specific alternative H 1 .
675
to statistical issues that arise in discovery-orientated analyses at the LHC. Ref
[74] deals with statistical issues that occur in Particle Physics searches for new
phenomena; as an example, it includes the successful search for the Higgs boson
at the LHC. A more detailed description of the plans for the Higgs search before its
discovery is in ref. [75].
15.7.1 H 0 , or H 0 Versus H 1 ?
In looking for new physics, there are two distinct types of approach. We can compare
our data just with the null hypothesis H 0 , the SM of Particle Physics; alternatively
we can see whether our data are more consistent with H 0 or with an alternative
hypothesis H 1 , some specific manifestation of new physics, such as a particular
form of quark and/or lepton substructure. The former is known as ‘goodness of fit’,
while the term ‘hypothesis testing’ is often reserved for the latter.
Each of these approaches has its own advantage. By not specifying a specific
alternative, 22 the goodness of fit test may be capable of detecting any form of
deviation from the SM. On the other hand, if we are searching for some specific
new effect, a comparison of H 0 and H 1 is likely to be a more sensitive way for that
particular alternative. Also, the ‘hypothesis testing’ approach is less likely to give a
false discovery claim if the assumed form of H 0 has been slightly mis-modelled.
15.7.2 p-Values
In order to quantify the chance of the observed effect being due to an uninteresting
statistical fluctuation, some statistic is chosen for the data. The simplest case would
be the observed number n 0 of interesting events. Then the p-value is calculated,
which is simply the probability that, given the expected background rate b from
known sources, the observed value would fluctuate up to n 0 or larger. In more
complicated examples involving several relevant observables, the data statistic may
be a likelihood ratio L 0 /L 1 for the likelihood of the null hypothesis H 0 compared
with that for a specific alternative H 1 .
To compute the p-value of the observed or of possible data, the distribution f (t)
of the data statistic t under the relevant hypothesis is required. In some cases this
can be obtained analytically, but in more complicated situations, f (t) may require
simulation. For t being −2 ln L 0 /L best , Cowan et al have given useful asymptotic
22 Even a test of the null hypothesis may not be completely independent of ideas about alternatives.
Thus in an event counting experiment, new physics usually results in an increase in rate, unless
we are looking for neutrino oscillations, in which case a decrease would be significant. Also,
sometimes the statistic used for a goodness of fit test of H 0 may be the likelihood ratio for H 0 as
compared with a specific alternative H 1 .