15 Statistical Issues in Particle Physics
671
To combine N different uncorrelated measurements a i ± σ i of the same physical
quantity a 18 when the measurements are believed to be Gaussian distributed about
the true value a true , the well-known result is that the best estimate a comb ± σ comb is
given by
a comb = (a i ∗ w i )//w i , σ comb = 1/
i ,
(15.13)
where the weights are defined as w i = 1/σ 2
i . This is readily derived from
minimising with respect to a a weighted sum of squared deviations
S(a) = (a i − a)
2 /σ
2
i
(15.14)
The extension to the case where the individual measurements are correlated
(as is often the case for analyses using different techniques on the same data)
is straightforward: S(a) becomes (a i − a) ∗ H ij ∗ (a j − a), where H is the
inverse covariance matrix for the a i . It provides Best Linear Unbiassed Estimates
(BLUE)[70].
There are, however, practical details that complicate its application. For example,
in the above formula, the σ i are supposed to be the true accuracies of the
measurements. Often, all that we have available are estimates of their values.
Problems arise in situations where the uncertainty estimate depends on the measured
value a i . For example, in counting experiments with Poisson statistics, it is typical
to set the uncertainty as the square root of the observed number. Then a downward
fluctuation in the observation results in an overestimated weight, and a comb is
biassed downwards. If instead the uncertainty is estimated as the square root of
the expected number a, the combined result is biassed upwards—the increased
uncertainty reduces S at larger a. A way round this difficulty has been suggested
by Lyons et al. [71]. Alternatively, for Poisson counting data a likelihood approach
is preferable to a χ 2 -based method.
Another problem arises when the individual measurements are very correlated.
When the correlation coefficient of two uncertainties is larger than σ 1 /σ 2 (where σ 1
is the smaller uncertainty), a comb lies outside the range of the two measurements.
As the correlation coefficient tends to +1, the extrapolation becomes larger, and is
sensitive to the exact values assumed for the elements of the covariance matrix. The
situation is aggravated by the fact that σ comb tends to zero. This is usually dealt with
by selecting one of the two analyses, rather than trying to combine them. However,
if the estimated uncertainty increases with the estimated value, choosing the result
with the smaller estimated uncertainty can again produce a downward bias. On the
other hand, using the smaller expected uncertainty can cause us to ignore an analysis
which had a particularly favourable statistical fluctuation, which produced a result
18 It is of course much better to use all the data in a combined analysis, rather than simply to
combine the results.
671
To combine N different uncorrelated measurements a i ± σ i of the same physical
quantity a 18 when the measurements are believed to be Gaussian distributed about
the true value a true , the well-known result is that the best estimate a comb ± σ comb is
given by
a comb = (a i ∗ w i )//w i , σ comb = 1/
i ,
(15.13)
where the weights are defined as w i = 1/σ 2
i . This is readily derived from
minimising with respect to a a weighted sum of squared deviations
S(a) = (a i − a)
2 /σ
2
i
(15.14)
The extension to the case where the individual measurements are correlated
(as is often the case for analyses using different techniques on the same data)
is straightforward: S(a) becomes (a i − a) ∗ H ij ∗ (a j − a), where H is the
inverse covariance matrix for the a i . It provides Best Linear Unbiassed Estimates
(BLUE)[70].
There are, however, practical details that complicate its application. For example,
in the above formula, the σ i are supposed to be the true accuracies of the
measurements. Often, all that we have available are estimates of their values.
Problems arise in situations where the uncertainty estimate depends on the measured
value a i . For example, in counting experiments with Poisson statistics, it is typical
to set the uncertainty as the square root of the observed number. Then a downward
fluctuation in the observation results in an overestimated weight, and a comb is
biassed downwards. If instead the uncertainty is estimated as the square root of
the expected number a, the combined result is biassed upwards—the increased
uncertainty reduces S at larger a. A way round this difficulty has been suggested
by Lyons et al. [71]. Alternatively, for Poisson counting data a likelihood approach
is preferable to a χ 2 -based method.
Another problem arises when the individual measurements are very correlated.
When the correlation coefficient of two uncertainties is larger than σ 1 /σ 2 (where σ 1
is the smaller uncertainty), a comb lies outside the range of the two measurements.
As the correlation coefficient tends to +1, the extrapolation becomes larger, and is
sensitive to the exact values assumed for the elements of the covariance matrix. The
situation is aggravated by the fact that σ comb tends to zero. This is usually dealt with
by selecting one of the two analyses, rather than trying to combine them. However,
if the estimated uncertainty increases with the estimated value, choosing the result
with the smaller estimated uncertainty can again produce a downward bias. On the
other hand, using the smaller expected uncertainty can cause us to ignore an analysis
which had a particularly favourable statistical fluctuation, which produced a result
18 It is of course much better to use all the data in a combined analysis, rather than simply to
combine the results.