15 Statistical Issues in Particle Physics
651
are prepared to use Bayesian techniques for multidimensional problems (e.g. with
systematics). They would, however, like to ensure that the technique they use
provides parameter intervals with reasonable frequentist coverage.
15.1.3.4 Coverage
One of the major advantages of the frequentist Neyman construction is that it
guarantees coverage. This is a property of a statistical technique 4 for calculating
intervals, and specifies how often the interval contains the true value μ t of the
parameter. This can vary with μ t .
For example, for a Poisson counting experiment with parameter μ and observed
number n, a (not very good) method for providing an interval for μ is n ±
√
n. Thus
an observed n = 2 would give a range 0.59–3.41 for μ. If μ = 2.01, observed
values n = 2, 3 and 4 result in intervals that include μ = 2.01, while other values
of n do not. The coverage of this procedure for μ = 2.01 is thus the sum of the
Poisson probabilities for having n = 2, 3 or 4 for the given μ.
For a discrete observable (e.g. the number of detected events in a search for Dark
Matter), there are jumps in the coverage; in order to avoid under-coverage, there
is necessarily some over-coverage. However, for a continuous observable (e.g. the
estimated mass of the Higgs boson) the coverage can be exact.
Coverage is not guaranteed for methods that do not use the Neyman construction
(see Sect. 15.2.1). Interesting plots of coverage as a function of the parameter value
for the simple case of a Poisson counting experiment can be found in ref. [32].
15.1.3.5 Likelihoods
The likelihood approach makes use of the probability density function (pdf ) for
observing the data, evaluated for the data actually observed. 5 It is a function of
any parameters, although it does not behave like a probability density for them. It
provides a method for determining values of parameters. These include point estimates for the ‘best’ values, and ranges (or contours in multi-parameter situations)
to characterise the uncertainties. It usually has good properties asymptotically, but a
major use is with sparse multi-dimensional data.
The likelihood method is neither frequentist nor Bayesian. It thus does not
guarantee frequentist coverage or Bayesian credibility. It does, however, play a
central role in the Bayesian approach, which obtains the posterior probability
4 It is important to realise that coverage is a property of the method, and not of an individual
measurement.
5 The pdf f (x, μ 0 ) gives the probability density for obtaining various data x when the parameter
has some specified value μ 0 . The likelihood is the same function of two variables f (x 0 , μ), but
now with x 0 fixed at the data actually obtained, and μ regarded as the variable.
651
are prepared to use Bayesian techniques for multidimensional problems (e.g. with
systematics). They would, however, like to ensure that the technique they use
provides parameter intervals with reasonable frequentist coverage.
15.1.3.4 Coverage
One of the major advantages of the frequentist Neyman construction is that it
guarantees coverage. This is a property of a statistical technique 4 for calculating
intervals, and specifies how often the interval contains the true value μ t of the
parameter. This can vary with μ t .
For example, for a Poisson counting experiment with parameter μ and observed
number n, a (not very good) method for providing an interval for μ is n ±
√
n. Thus
an observed n = 2 would give a range 0.59–3.41 for μ. If μ = 2.01, observed
values n = 2, 3 and 4 result in intervals that include μ = 2.01, while other values
of n do not. The coverage of this procedure for μ = 2.01 is thus the sum of the
Poisson probabilities for having n = 2, 3 or 4 for the given μ.
For a discrete observable (e.g. the number of detected events in a search for Dark
Matter), there are jumps in the coverage; in order to avoid under-coverage, there
is necessarily some over-coverage. However, for a continuous observable (e.g. the
estimated mass of the Higgs boson) the coverage can be exact.
Coverage is not guaranteed for methods that do not use the Neyman construction
(see Sect. 15.2.1). Interesting plots of coverage as a function of the parameter value
for the simple case of a Poisson counting experiment can be found in ref. [32].
15.1.3.5 Likelihoods
The likelihood approach makes use of the probability density function (pdf ) for
observing the data, evaluated for the data actually observed. 5 It is a function of
any parameters, although it does not behave like a probability density for them. It
provides a method for determining values of parameters. These include point estimates for the ‘best’ values, and ranges (or contours in multi-parameter situations)
to characterise the uncertainties. It usually has good properties asymptotically, but a
major use is with sparse multi-dimensional data.
The likelihood method is neither frequentist nor Bayesian. It thus does not
guarantee frequentist coverage or Bayesian credibility. It does, however, play a
central role in the Bayesian approach, which obtains the posterior probability
4 It is important to realise that coverage is a property of the method, and not of an individual
measurement.
5 The pdf f (x, μ 0 ) gives the probability density for obtaining various data x when the parameter
has some specified value μ 0 . The likelihood is the same function of two variables f (x 0 , μ), but
now with x 0 fixed at the data actually obtained, and μ regarded as the variable.