15 Statistical Issues in Particle Physics
673
15.5.3 Data Consistency
The standard procedure for combining data pays no attention to whether or not
the data are consistent. If they are clearly inconsistent, then they should not all
be combined. When they are somewhat inconsistent, the procedure adopted by the
Particle Data Group[14] is to increase all the uncertainties by a common factor such
that the overall χ 2 per degree of freedom equals unity. 20
The Particle Data Group prescription for expanding uncertainties in the case of
discrepant data sets has complications when each of the data sets consists of two or
more parameters[72].
15.6 Goodness of Fit
15.6.1 Sparse Multi-Dimensional Data
The standard method loved by most scientists uses the weighted sum of squares,
commonly called χ 2 . This, however, is only applicable to binned data (i.e. in a one
or more dimensional histogram). Furthermore it loses its attractive feature that its
distribution is model-independent when there is not enough data, which is likely to
be so in the multi-dimensional case.
Although the maximum likelihood method is very useful for parameter determination with unbinned data, the value of L max usually does not provide a measure
of goodness of fit (see Sect. 15.2.2).
An alternative that is used for sparse one-dimensional data is the KolmogorovSmirnov (KS) approach[68], or one of its variants. However, in the presence of fitted
parameters, simulation is again required to determine the expected distribution of
the KS-distance. Also because of the problem of how to order the data, the way to
use it in multi-dimensional situations is not unique.
The standard KS method uses the maximum deviation between two cumulative
distributions; because of statistical fluctuations, this is likely to occur near the
middle of the distributions. In cases where interesting New Physics is expected to
occur at extreme values of some kinematic variable (e.g. p T ), variants of KS such as
Anderson-Darling[69] that give extra weight to the distributions’ tails may be more
useful.
20 This is somewhat conservative, in that even if there are no problems, about half the data sets
would be expected to have this larger than unity.
673
15.5.3 Data Consistency
The standard procedure for combining data pays no attention to whether or not
the data are consistent. If they are clearly inconsistent, then they should not all
be combined. When they are somewhat inconsistent, the procedure adopted by the
Particle Data Group[14] is to increase all the uncertainties by a common factor such
that the overall χ 2 per degree of freedom equals unity. 20
The Particle Data Group prescription for expanding uncertainties in the case of
discrepant data sets has complications when each of the data sets consists of two or
more parameters[72].
15.6 Goodness of Fit
15.6.1 Sparse Multi-Dimensional Data
The standard method loved by most scientists uses the weighted sum of squares,
commonly called χ 2 . This, however, is only applicable to binned data (i.e. in a one
or more dimensional histogram). Furthermore it loses its attractive feature that its
distribution is model-independent when there is not enough data, which is likely to
be so in the multi-dimensional case.
Although the maximum likelihood method is very useful for parameter determination with unbinned data, the value of L max usually does not provide a measure
of goodness of fit (see Sect. 15.2.2).
An alternative that is used for sparse one-dimensional data is the KolmogorovSmirnov (KS) approach[68], or one of its variants. However, in the presence of fitted
parameters, simulation is again required to determine the expected distribution of
the KS-distance. Also because of the problem of how to order the data, the way to
use it in multi-dimensional situations is not unique.
The standard KS method uses the maximum deviation between two cumulative
distributions; because of statistical fluctuations, this is likely to occur near the
middle of the distributions. In cases where interesting New Physics is expected to
occur at extreme values of some kinematic variable (e.g. p T ), variants of KS such as
Anderson-Darling[69] that give extra weight to the distributions’ tails may be more
useful.
20 This is somewhat conservative, in that even if there are no problems, about half the data sets
would be expected to have this larger than unity.