Joint and Conditional Probabilities
29
The numerator uses the original joint probability, but this equality can be considered
as a renormalization of it due to the fixing of d. If you are familiar with statistical
mechanics of spins under an external magnetic field, this concept can be understood
by considering the spin degrees of freedom as x and the external magnetic field as
d. By the way, even if the roles of x and d are exchanged, the same holds for the
same reason:
P (d|x) =
P (x, d)
P (x)
=
P (x, d)
D P (x, D)
.
(2.31)
Note that the numerator on the right-hand side is common between (2.30) and (2.31),
P (x|d)P (d) = P (d|x)P (x) = P (x, d) .
(2.32)
This is called Bayes’ theorem and is often used in this book.
Monty Hall problem and conditional probability
Here is a notorious problem of probability theory. Suppose only one of three boxes
has a prize. Let the contents of each box be x 1 , x 2 , x 3 . Possible combinations are
(x 1 = ◦, x 2 = ×, x 3 = ×),
(x 1 = ×, x 2 = ◦, x 3 = ×),
(x 1 = ×, x 2 = ×, x 3 = ◦).
(2.33)
So it is a good idea to take the probability of 1/3 for each.
P (x 1 , x 2 , x 3 ) =
1
3
.
(2.34)
Then you pick any box. At this stage, all boxes are equal, so suppose you pick x 1 .
The probability of finding the prize is 1/3, the probability of not finding it is 2/3:
P (x 1 = ◦) =
1
3
, P(x 1 = ×) =
2
3
.
(2.35)
This is just a marginal probability because you have summed the probability other
than x 1 .
Now, suppose you can’t open the x 1 box. Instead, there is a moderator, who
knows the contents of every box. The moderator opens one of the boxes x 2 , x 3 and
shows to you that there is nothing inside. Then, all that remain are x 1 , which you
chose, and one of x 2 , x 3 , which the moderator did not open. Should you stay at x 1
or choose the box that the moderator did not open? This is the Monty Hall problem.
Let us solve this problem taking into account conditional probabilities. First,
P (x 2 or x 3 = |x 1 = ◦) = 0.
(2.36)
29
The numerator uses the original joint probability, but this equality can be considered
as a renormalization of it due to the fixing of d. If you are familiar with statistical
mechanics of spins under an external magnetic field, this concept can be understood
by considering the spin degrees of freedom as x and the external magnetic field as
d. By the way, even if the roles of x and d are exchanged, the same holds for the
same reason:
P (d|x) =
P (x, d)
P (x)
=
P (x, d)
D P (x, D)
.
(2.31)
Note that the numerator on the right-hand side is common between (2.30) and (2.31),
P (x|d)P (d) = P (d|x)P (x) = P (x, d) .
(2.32)
This is called Bayes’ theorem and is often used in this book.
Monty Hall problem and conditional probability
Here is a notorious problem of probability theory. Suppose only one of three boxes
has a prize. Let the contents of each box be x 1 , x 2 , x 3 . Possible combinations are
(x 1 = ◦, x 2 = ×, x 3 = ×),
(x 1 = ×, x 2 = ◦, x 3 = ×),
(x 1 = ×, x 2 = ×, x 3 = ◦).
(2.33)
So it is a good idea to take the probability of 1/3 for each.
P (x 1 , x 2 , x 3 ) =
1
3
.
(2.34)
Then you pick any box. At this stage, all boxes are equal, so suppose you pick x 1 .
The probability of finding the prize is 1/3, the probability of not finding it is 2/3:
P (x 1 = ◦) =
1
3
, P(x 1 = ×) =
2
3
.
(2.35)
This is just a marginal probability because you have summed the probability other
than x 1 .
Now, suppose you can’t open the x 1 box. Instead, there is a moderator, who
knows the contents of every box. The moderator opens one of the boxes x 2 , x 3 and
shows to you that there is nothing inside. Then, all that remain are x 1 , which you
chose, and one of x 2 , x 3 , which the moderator did not open. Should you stay at x 1
or choose the box that the moderator did not open? This is the Monty Hall problem.
Let us solve this problem taking into account conditional probabilities. First,
P (x 2 or x 3 = |x 1 = ◦) = 0.
(2.36)