674
L. Lyons
15.6.2 Number of Degrees of Freedom
If we construct the weighted sum of squares S between a predicted theoretical curve
and some data in the form of a histogram, provided the Poisson distribution of the
bin contents can be approximated by a Gaussian (and the theory is correct, the data
are unbiased, the uncertainty estimates are correct, etc.), asymptotically 21 S will
be distributed as χ 2 with the number of degrees of freedom ν = n − f , where n is
the number of data points and f is the number of free parameters whose values are
determined by minimising S.
The relevance of the asymptotic requirement can be seen by imagining fitting a
more or less flat distribution by the expression N(1 + 10 −6 cos(x − x 0 )), where the
free parameters are the normalisation N and the phase x 0 . It is clear that, although
x 0 is left free in the fit, because of the 10 −6 factor, it will have a negligible effect
on the fitted curve, and hence will not result in the typical reduction in S associated
with having an extra free parameter. Of course, with an enormous amount of data,
we would have sensitivity to x 0 , and so asymptotically it does reduce ν by one unit,
but not for smaller amounts of data.
Another example involves neutrino oscillation experiments[54]. In a simplified
two neutrino scenario, the neutrino energy spectrum is fitted by a survival probability P of the form
P = 1 − sin
2 2θ sin
2 (C ∗
2 ),
(15.15)
where C is a known function of the neutrino energy and the length of its flight path,
2 is the difference in mass squared of the relevant neutrino species, and θ is the
neutrino mixing angle. For small values of C ∗ 2 , this reduces to
P ≈ 1 − sin
2 2θ (C ∗
2 )
2
(15.16)
Thus the survival probability depends on the two parameters only via their product
sin 2θ θm 2 . Because this combination is all that we can hope to determine, we
effectively have only one free parameter rather than two. Of course, an enormous
amount of data can manage to distinguish between sin
C ∗ 2
and C ∗ 2 , and
so asymptotically we have two free parameters as expected.
15.7 Discovery Issues
Searches for new particles are an exciting endeavour, and continue to play a large
role at the LHC at CERN, in neutrino experiments, in searches for dark matter, etc.
The 2007 and 2011 PHYSTAT Workshops at CERN[7, 9] were devoted specifically
21 The examples in this section go beyond the requirement that we need enough events for the
Poisson distribution to be well approximated by a Gaussian.
L. Lyons
15.6.2 Number of Degrees of Freedom
If we construct the weighted sum of squares S between a predicted theoretical curve
and some data in the form of a histogram, provided the Poisson distribution of the
bin contents can be approximated by a Gaussian (and the theory is correct, the data
are unbiased, the uncertainty estimates are correct, etc.), asymptotically 21 S will
be distributed as χ 2 with the number of degrees of freedom ν = n − f , where n is
the number of data points and f is the number of free parameters whose values are
determined by minimising S.
The relevance of the asymptotic requirement can be seen by imagining fitting a
more or less flat distribution by the expression N(1 + 10 −6 cos(x − x 0 )), where the
free parameters are the normalisation N and the phase x 0 . It is clear that, although
x 0 is left free in the fit, because of the 10 −6 factor, it will have a negligible effect
on the fitted curve, and hence will not result in the typical reduction in S associated
with having an extra free parameter. Of course, with an enormous amount of data,
we would have sensitivity to x 0 , and so asymptotically it does reduce ν by one unit,
but not for smaller amounts of data.
Another example involves neutrino oscillation experiments[54]. In a simplified
two neutrino scenario, the neutrino energy spectrum is fitted by a survival probability P of the form
P = 1 − sin
2 2θ sin
2 (C ∗
2 ),
(15.15)
where C is a known function of the neutrino energy and the length of its flight path,
2 is the difference in mass squared of the relevant neutrino species, and θ is the
neutrino mixing angle. For small values of C ∗ 2 , this reduces to
P ≈ 1 − sin
2 2θ (C ∗
2 )
2
(15.16)
Thus the survival probability depends on the two parameters only via their product
sin 2θ θm 2 . Because this combination is all that we can hope to determine, we
effectively have only one free parameter rather than two. Of course, an enormous
amount of data can manage to distinguish between sin
C ∗ 2
and C ∗ 2 , and
so asymptotically we have two free parameters as expected.
15.7 Discovery Issues
Searches for new particles are an exciting endeavour, and continue to play a large
role at the LHC at CERN, in neutrino experiments, in searches for dark matter, etc.
The 2007 and 2011 PHYSTAT Workshops at CERN[7, 9] were devoted specifically
21 The examples in this section go beyond the requirement that we need enough events for the
Poisson distribution to be well approximated by a Gaussian.