648
L. Lyons
in repeated time intervals do follow a Poisson distribution; etc.), and hence we
want to use a statistical approach that allows the data “to speak for themselves”,
rather than our analysis being sensitive to our assumptions and beliefs, as embodied
in the assumed Bayesian priors. Bayesians would counter this by remarking that
frequentist inference can depend on the reference ensemble, the ordering rule, the
stopping rule, etc.
With enough data, the results of Bayesian and frequentist approaches usually
tend to agree. However, in smallish data samples numerical results from the two
approaches can differ.
15.1.3.1 Probability
There are at least three different approaches to the question of what probability is.
The first is the mathematical one, which is based on axioms e.g. it must lie in the
range 0–1; the probabilities of an event occurring and of it not occurring add up to
1; etc. It does not give much feeling for what probability is, but it does provide the
underpinning for the next two methods.
Frequentists, not surprisingly, define probability in terms of frequencies in a
long series of essentially identical repetitions 2 of the relevant procedure. Thus the
probability of the number 5 being uppermost in throws of a die is 1/6, because that
is the fraction of times we expect (or approximately observe) it to happen. This
implies that probability cannot be defined for a specific occurrence (Will the first
astronaut who lands on Mars return to earth alive?) or for the value of a physical
constant (Does Dark Matter contribute more than 25% of the critical density of the
Universe?).
In contrast, Bayesians define probability in terms of degree of belief. Thus
it can be used for unique events or for the values of physical constants. It can
also vary from person to person, because my information may differ from yours.
The numerical value of the probability to be assigned to a particular statement is
determined by the concept of a ‘fair bet’; if I think the probability (or ‘Bayesian
credibility’) of the statement being true is 20%, then I must offer odds of 4-to-1, and
allow you to bet in either direction.
This difference in approach to probability affects the way Bayesians and
frequentists deal with statistical procedures. This is illustrated below by considering
parameter determination.
15.1.3.2 Bayesian Approach
The Bayesian approach makes use of Bayes’ Theorem:
p(A|B) = p(B|A) × p(A)/p(B),
(15.1)
2 Bayesians attack this concept of ‘essentially identical trials’, claiming that it is hard to define it
without using the concept of probability, thus making the definition circular.
L. Lyons
in repeated time intervals do follow a Poisson distribution; etc.), and hence we
want to use a statistical approach that allows the data “to speak for themselves”,
rather than our analysis being sensitive to our assumptions and beliefs, as embodied
in the assumed Bayesian priors. Bayesians would counter this by remarking that
frequentist inference can depend on the reference ensemble, the ordering rule, the
stopping rule, etc.
With enough data, the results of Bayesian and frequentist approaches usually
tend to agree. However, in smallish data samples numerical results from the two
approaches can differ.
15.1.3.1 Probability
There are at least three different approaches to the question of what probability is.
The first is the mathematical one, which is based on axioms e.g. it must lie in the
range 0–1; the probabilities of an event occurring and of it not occurring add up to
1; etc. It does not give much feeling for what probability is, but it does provide the
underpinning for the next two methods.
Frequentists, not surprisingly, define probability in terms of frequencies in a
long series of essentially identical repetitions 2 of the relevant procedure. Thus the
probability of the number 5 being uppermost in throws of a die is 1/6, because that
is the fraction of times we expect (or approximately observe) it to happen. This
implies that probability cannot be defined for a specific occurrence (Will the first
astronaut who lands on Mars return to earth alive?) or for the value of a physical
constant (Does Dark Matter contribute more than 25% of the critical density of the
Universe?).
In contrast, Bayesians define probability in terms of degree of belief. Thus
it can be used for unique events or for the values of physical constants. It can
also vary from person to person, because my information may differ from yours.
The numerical value of the probability to be assigned to a particular statement is
determined by the concept of a ‘fair bet’; if I think the probability (or ‘Bayesian
credibility’) of the statement being true is 20%, then I must offer odds of 4-to-1, and
allow you to bet in either direction.
This difference in approach to probability affects the way Bayesians and
frequentists deal with statistical procedures. This is illustrated below by considering
parameter determination.
15.1.3.2 Bayesian Approach
The Bayesian approach makes use of Bayes’ Theorem:
p(A|B) = p(B|A) × p(A)/p(B),
(15.1)
2 Bayesians attack this concept of ‘essentially identical trials’, claiming that it is hard to define it
without using the concept of probability, thus making the definition circular.