676
L. Lyons
formulae for f (t)[76]; here L best is the value of the likelihood when the parameters
in H 0 are set at their best values.
A small value of p indicates that the data are not very compatible with the theory
(which may be because the detector’s response or the background is poorly modeled,
rather than the theory being wrong).
Particle Physicists usually convert p into the number of standard deviations σ
of a Gaussian distribution, beyond which the one-sided tail area corresponds to p;
statisticians refer to this as the z-score, but physicists call it significance. Thus 5σ
corresponds to a p-value of 3 ∗ 10 −7 . This is done simply because it provides a
number which is easier to remember, and not because Gaussians are relevant for
every situation.
Unfortunately, p-values are often misinterpreted as the probability of the theory
being true, given the data. It sometimes helps colleagues clarify the difference
between p(A|B) and p(B|A) by reminding them that the probability of being
pregnant, given the fact that you are female, is considerably smaller than the
probability of being female, given the fact that you are pregnant. Reference [77]
contains a series of articles by statisticians on the use (and misuse) of p-values.
Sometimes S/
√
B or S/
√
(S + B) or the like (where S is the number of observed
events above the estimated background B) is used as an approximate measure of
significance. These approximations can be very poor, and their use is in general not
recommended. 23
15.7.3 CL s
This is a technique[58] which is used for situations in which a discovery is not
made, and instead various parameter values are excluded. For example the failure to
observe SUSY particles can be converted into mass ranges which are excluded (at
some confidence level).
Figure 15.5 (again) illustrates the expected distributions for some suitably chosen
statistic t under two different hypotheses: the null H 0 in which there is only standard
known physics, and H 1 which also includes some specific new particle, such as
a SUSY neutralino. In Fig. 15.5c, the new particle is produced prolifically, and
an experimental observation of t should fall in one peak or the other, and easily
distinguishes between the two hypotheses. In contrast, Fig. 15.5a corresponds to
very weak production of the new particle and it is almost impossible to know
whether the new particle is being produced or not.
23 For example, if selections to enhance signal with respect to background were optimised using
S/
√
B, extremely hard cuts might be chosen, yielding expected numbers of events S = 0.1 and
B = 10 −3 . This results in S/
√
B = 10, which sounds very good, but in fact this selection is
disastrous.
L. Lyons
formulae for f (t)[76]; here L best is the value of the likelihood when the parameters
in H 0 are set at their best values.
A small value of p indicates that the data are not very compatible with the theory
(which may be because the detector’s response or the background is poorly modeled,
rather than the theory being wrong).
Particle Physicists usually convert p into the number of standard deviations σ
of a Gaussian distribution, beyond which the one-sided tail area corresponds to p;
statisticians refer to this as the z-score, but physicists call it significance. Thus 5σ
corresponds to a p-value of 3 ∗ 10 −7 . This is done simply because it provides a
number which is easier to remember, and not because Gaussians are relevant for
every situation.
Unfortunately, p-values are often misinterpreted as the probability of the theory
being true, given the data. It sometimes helps colleagues clarify the difference
between p(A|B) and p(B|A) by reminding them that the probability of being
pregnant, given the fact that you are female, is considerably smaller than the
probability of being female, given the fact that you are pregnant. Reference [77]
contains a series of articles by statisticians on the use (and misuse) of p-values.
Sometimes S/
√
B or S/
√
(S + B) or the like (where S is the number of observed
events above the estimated background B) is used as an approximate measure of
significance. These approximations can be very poor, and their use is in general not
recommended. 23
15.7.3 CL s
This is a technique[58] which is used for situations in which a discovery is not
made, and instead various parameter values are excluded. For example the failure to
observe SUSY particles can be converted into mass ranges which are excluded (at
some confidence level).
Figure 15.5 (again) illustrates the expected distributions for some suitably chosen
statistic t under two different hypotheses: the null H 0 in which there is only standard
known physics, and H 1 which also includes some specific new particle, such as
a SUSY neutralino. In Fig. 15.5c, the new particle is produced prolifically, and
an experimental observation of t should fall in one peak or the other, and easily
distinguishes between the two hypotheses. In contrast, Fig. 15.5a corresponds to
very weak production of the new particle and it is almost impossible to know
whether the new particle is being produced or not.
23 For example, if selections to enhance signal with respect to background were optimised using
S/
√
B, extremely hard cuts might be chosen, yielding expected numbers of events S = 0.1 and
B = 10 −3 . This results in S/
√
B = 10, which sounds very good, but in fact this selection is
disastrous.