15 Statistical Issues in Particle Physics
667
true, and deviations are likely to be most serious in the tails of the distribution. A
log-normal or gamma function may be a better choice.
There are many methods for incorporating nuisance parameters in upper limit
calculations. These include:
• Profile likelihood (see also Sect. 15.2.3)
The likelihood, based on the data from the main and from the subsidiary
measurements, is a function of the parameter of interest s and of the nuisance parameters. The profile likelihood L prof (s) is simply the full likelihood
L(s, b best (s), , best (s)), evaluated at the values of the nuisance parameters that
maximise the likelihood at each s. Then the profile likelihood is simply used to
extract the limits on s, much as the ordinary likelihood could be used for the case
when there are no nuisance parameters.
Rolke et al. [59] have studied the behaviour of the profile likelihood method
for limits. Heinrich[32] had shown that the likelihood approach for estimating
a Poisson parameter (in the absence of both background and of nuisance
parameters) can have poor coverage at low values of the Poisson parameter.
However, the profile likelihood seems to do better, probably because the nuisance
parameters have the effect of smoothing away the fluctuating coverage observed
by Heinrich.
• Fully Bayesian
When there is a subsidiary measurement for a nuisance parameter, a prior
is chosen for b (or ), the data are used to extract the likelihood, and then
Bayes’ Theorem is used to deduce the posterior for the nuisance parameter.
This posterior from the subsidiary measurement is then used as the prior for the
nuisance parameter in the main measurement (this prior could alternatively come
from information other than a subsidiary measurement); with the prior for s and
the likelihood for the main measurement, the overall joint posterior for s and
the nuisance parameter(s) is derived. 17 This is then integrated over the nuisance
parameter(s) to determine the posterior for s, from which an upper limit can be
derived; this procedure is known as marginalisation.
Numerical examples of upper limits can be found in ref. [60], where a method
is discussed in detail. Thus assuming (somewhat unrealistically) precisely determined backgrounds, the effect of a 10% uncertainty in can be seen for various
measured values of n in Table 15.1. A plot of the coverage when the uncertainty
in is 20% is reproduced in Fig. 15.6.
It is not universally appreciated that the choice for the main measurement of a
truncated Gaussian prior for and an (improper) constant prior for non-negative
s results in a posterior for s which diverges[61]. Thus numerical estimates of the
relevant integrals are meaningless. Another problem comes from the difficulty
of choosing sensible multi-dimensional priors. Heinrich has pointed out the
17 This is usually equivalent to starting with a prior for s and the nuisance parameters, and the
likelihood for the data from the main and the subsidiary experiments together, to obtain the joint
posterior.
667
true, and deviations are likely to be most serious in the tails of the distribution. A
log-normal or gamma function may be a better choice.
There are many methods for incorporating nuisance parameters in upper limit
calculations. These include:
• Profile likelihood (see also Sect. 15.2.3)
The likelihood, based on the data from the main and from the subsidiary
measurements, is a function of the parameter of interest s and of the nuisance parameters. The profile likelihood L prof (s) is simply the full likelihood
L(s, b best (s), , best (s)), evaluated at the values of the nuisance parameters that
maximise the likelihood at each s. Then the profile likelihood is simply used to
extract the limits on s, much as the ordinary likelihood could be used for the case
when there are no nuisance parameters.
Rolke et al. [59] have studied the behaviour of the profile likelihood method
for limits. Heinrich[32] had shown that the likelihood approach for estimating
a Poisson parameter (in the absence of both background and of nuisance
parameters) can have poor coverage at low values of the Poisson parameter.
However, the profile likelihood seems to do better, probably because the nuisance
parameters have the effect of smoothing away the fluctuating coverage observed
by Heinrich.
• Fully Bayesian
When there is a subsidiary measurement for a nuisance parameter, a prior
is chosen for b (or ), the data are used to extract the likelihood, and then
Bayes’ Theorem is used to deduce the posterior for the nuisance parameter.
This posterior from the subsidiary measurement is then used as the prior for the
nuisance parameter in the main measurement (this prior could alternatively come
from information other than a subsidiary measurement); with the prior for s and
the likelihood for the main measurement, the overall joint posterior for s and
the nuisance parameter(s) is derived. 17 This is then integrated over the nuisance
parameter(s) to determine the posterior for s, from which an upper limit can be
derived; this procedure is known as marginalisation.
Numerical examples of upper limits can be found in ref. [60], where a method
is discussed in detail. Thus assuming (somewhat unrealistically) precisely determined backgrounds, the effect of a 10% uncertainty in can be seen for various
measured values of n in Table 15.1. A plot of the coverage when the uncertainty
in is 20% is reproduced in Fig. 15.6.
It is not universally appreciated that the choice for the main measurement of a
truncated Gaussian prior for and an (improper) constant prior for non-negative
s results in a posterior for s which diverges[61]. Thus numerical estimates of the
relevant integrals are meaningless. Another problem comes from the difficulty
of choosing sensible multi-dimensional priors. Heinrich has pointed out the
17 This is usually equivalent to starting with a prior for s and the nuisance parameters, and the
likelihood for the data from the main and the subsidiary experiments together, to obtain the joint
posterior.