15 Statistical Issues in Particle Physics
687
15.9.5 Non-parametric Methods
These are so unknown to most Particle Physicists that they are usually unaware
when they are using them. Simple examples include:
• A histogramme as an estimate of the density distribution of a variable of interest.
• Kernel density estimation.
• Kolmogorov-Smirnov or Anderson-Darling methods, to test whether distributions are consistent.
• Classification schemes based on k nearest neighbours.
• Neural networks
These all avoid the need to specify a particular parametric form, and hence
the values of any parameters. In general such a method is less powerful than a
parametric one, if the latter were available and relevant.
15.9.6 Collaboration with Statisticians
Other scientists seem to be better than particle physicists about involving statisticians in the analysis of their data. This is partly due to the fact that we like to try
out statistical techniques ourselves; that we consider our data is too complicated for
other people to deal with; and that we are somewhat over-protective of our data,
and are reluctant to share it with others. None of this is particularly convincing, and
it is clear that we would benefit from the involvement of professional statisticians.
The advantages of having them participating in the recent PHYSTAT meetings have
been obvious.
In the past, Particle Physicists have on occasion asked rather specific questions
to Statisticians they happened to know. Statisticians prefer to be much more directly
involved with the data itself. With analyses becoming more and more complex, it
will be highly desirable for them to be affiliated with experimental groups.
15.10 Conclusion
Although the statistical aspects of many particle physics analyses are already at a
sophisticated level, it is clear that there are many practical statistical issues to be
resolved. With the increasing complexity of scientific investigations, more active
collaboration with statisticians and machine learning experts will result in a better
understanding of the relevant techniques and improved analyses in the future.
Acknowledgements I wish to acknowledge the patience and expertise of David Cox, Brad Efron,
Jerry Friedman and David van Dyk, and also of other Statisticians too numerous to list, in
explaining statistical issues to me; the ones who have contributed to the PHYSTAT meetings have
687
15.9.5 Non-parametric Methods
These are so unknown to most Particle Physicists that they are usually unaware
when they are using them. Simple examples include:
• A histogramme as an estimate of the density distribution of a variable of interest.
• Kernel density estimation.
• Kolmogorov-Smirnov or Anderson-Darling methods, to test whether distributions are consistent.
• Classification schemes based on k nearest neighbours.
• Neural networks
These all avoid the need to specify a particular parametric form, and hence
the values of any parameters. In general such a method is less powerful than a
parametric one, if the latter were available and relevant.
15.9.6 Collaboration with Statisticians
Other scientists seem to be better than particle physicists about involving statisticians in the analysis of their data. This is partly due to the fact that we like to try
out statistical techniques ourselves; that we consider our data is too complicated for
other people to deal with; and that we are somewhat over-protective of our data,
and are reluctant to share it with others. None of this is particularly convincing, and
it is clear that we would benefit from the involvement of professional statisticians.
The advantages of having them participating in the recent PHYSTAT meetings have
been obvious.
In the past, Particle Physicists have on occasion asked rather specific questions
to Statisticians they happened to know. Statisticians prefer to be much more directly
involved with the data itself. With analyses becoming more and more complex, it
will be highly desirable for them to be affiliated with experimental groups.
15.10 Conclusion
Although the statistical aspects of many particle physics analyses are already at a
sophisticated level, it is clear that there are many practical statistical issues to be
resolved. With the increasing complexity of scientific investigations, more active
collaboration with statisticians and machine learning experts will result in a better
understanding of the relevant techniques and improved analyses in the future.
Acknowledgements I wish to acknowledge the patience and expertise of David Cox, Brad Efron,
Jerry Friedman and David van Dyk, and also of other Statisticians too numerous to list, in
explaining statistical issues to me; the ones who have contributed to the PHYSTAT meetings have