256
13 Solutions for Selected Exercises
Exercise 12.1
Inserting p i = 1/n for all i in (12.2), we find
H = −
n
i=1
1
n
log 2 (1/n) = −
1
n
log 2 (1/n)
n
i=1
1 = − log 2 (1/n)
(13.72)
because the sum evaluates to n.
Exercise 12.2
The probabilities p x (1) = 1 − α and p x (0) = α are shown in Fig. 12.2 and the
conditional probabilities p yx (y j |x i ) in (12.8), which allows us to calculate the joint
probabilities
p xy (x i , y j ) =
p xy (1, 1) p xy (0, 1)
p xy (1, 0) p xy (0, 0)
=
p yx (1|1) p x (1) p yx (1|0) p x (0)
p yx (0|1) p x (1) p yx (0|0) p x (0)
=
(1 − ε)(1 − α)
εα
ε(1 − α)
(1 − ε)α)
.
(13.73)
Summing the joint probabilities p xy (x i , y j ) over the different x i gives us
p y (y j ) =
p y (1)
p y (0)
=
x i
p xy (x i , 1)
x i
p xy (x i , 0)
=
p xy (1, 1) + p xy (0, 1)
p xy (1, 0) + p xy (0, 0)
=
1 − ε − α + 2εα
ε + α − 2εα
=
1 − γ
γ
(13.74)
with γ = ε + α − 2εα. The backwards conditional probabilities p xy (x i |y j ) follow
from Bayes’ theorem (12.12). Inserting the respective probabilities on the right-hand
side, we find
p xy (1|1) =
p yx (1|1) p x (1)
p y (1)
=
(1 − ε)(1 − α)
1 − ε − α + 2εα
p xy (1|0) =
p yx (0|1) p x (1)
p y (0)
=
ε(1 − α)
ε + α − 2εα
p xy (0|1) =
p yx (1|0) p x (0)
p y (1)
=
εα
1 − ε − α + 2εα
(13.75)
p xy (0|0) =
p yx (0|0) p x (0)
p y (0)
=
(1 − ε)α
ε + α − 2εα
.
Exercise 12.3
With the abbreviation H b (x) = −x log 2 (x) − (1 − x) log 2 (1 − x) for the entropy of
a binary system, and using the probabilities from the previous exercise, we find
13 Solutions for Selected Exercises
Exercise 12.1
Inserting p i = 1/n for all i in (12.2), we find
H = −
n
i=1
1
n
log 2 (1/n) = −
1
n
log 2 (1/n)
n
i=1
1 = − log 2 (1/n)
(13.72)
because the sum evaluates to n.
Exercise 12.2
The probabilities p x (1) = 1 − α and p x (0) = α are shown in Fig. 12.2 and the
conditional probabilities p yx (y j |x i ) in (12.8), which allows us to calculate the joint
probabilities
p xy (x i , y j ) =
p xy (1, 1) p xy (0, 1)
p xy (1, 0) p xy (0, 0)
=
p yx (1|1) p x (1) p yx (1|0) p x (0)
p yx (0|1) p x (1) p yx (0|0) p x (0)
=
(1 − ε)(1 − α)
εα
ε(1 − α)
(1 − ε)α)
.
(13.73)
Summing the joint probabilities p xy (x i , y j ) over the different x i gives us
p y (y j ) =
p y (1)
p y (0)
=
x i
p xy (x i , 1)
x i
p xy (x i , 0)
=
p xy (1, 1) + p xy (0, 1)
p xy (1, 0) + p xy (0, 0)
=
1 − ε − α + 2εα
ε + α − 2εα
=
1 − γ
γ
(13.74)
with γ = ε + α − 2εα. The backwards conditional probabilities p xy (x i |y j ) follow
from Bayes’ theorem (12.12). Inserting the respective probabilities on the right-hand
side, we find
p xy (1|1) =
p yx (1|1) p x (1)
p y (1)
=
(1 − ε)(1 − α)
1 − ε − α + 2εα
p xy (1|0) =
p yx (0|1) p x (1)
p y (0)
=
ε(1 − α)
ε + α − 2εα
p xy (0|1) =
p yx (1|0) p x (0)
p y (1)
=
εα
1 − ε − α + 2εα
(13.75)
p xy (0|0) =
p yx (0|0) p x (0)
p y (0)
=
(1 − ε)α
ε + α − 2εα
.
Exercise 12.3
With the abbreviation H b (x) = −x log 2 (x) − (1 − x) log 2 (1 − x) for the entropy of
a binary system, and using the probabilities from the previous exercise, we find