22
2 Elements of Quantum Information Theory
h( p) := − p log p − (1 − p) log(1 − p)
(2.46)
Given two random variables X and Y jointly distributed according to { p(x, y)}, the
joint Shannon entropy reads:
H (XY ) = −p(x, y)
x,y
p(x, y),
(2.47)
while the conditional entropy is defined as:
H (X |Y ) = H (XY ) − H (Y ).
(2.48)
The conditional entropy of X given Y quantifies how uncertain we are about X , given
that we learned the value of Y . Finally, the mutual information H (X : Y ) measures
the amount of information we gain on X by observing the value of Y . This is given
by the total amount of information of X , H (X ), minus the uncertainty that we still
have on X after learning Y , i.e. H (X |Y ). Thus we have:
H (X : Y ) = H (X ) − H (X |Y ) = H (X ) + H (Y ) − H (XY ).
(2.49)
Of the many properties satisfied by the above-defined entropies, we highlight in
particular that: H (X |Y ) = H (XY ) − H (Y ) ≥ 0. We could intuitively expect this,
since the uncertainty on both random variables X and Y must be greater than the
uncertainty on Y . Conversely, this does not hold in general for quantum states, whose
uncertainty is quantified by the von Neumann entropy.
Definition 2.7 (von Neumann entropy) The von Neumann entropy of a quantum
state ρ, with eigenvalues {λ i }, is defined as:
H (ρ) = − Tr[ρ log ρ] = −
i
λ i log λ i .
(2.50)
Often, the von Neumann entropy of a system A in state ρ is indicated as: H (A) ρ .
One can interpret the von Neumann entropy of ρ as the Shannon entropy of the
probability distribution defined by its eigenvalues, hence we use the same symbol.
For this analogy, the previous definitions of joint entropy (2.47), conditional entropy
(2.48) and mutual information (2.49) can be extended to the von Neumann entropy.
We observe that 0 ≤ H (ρ) ≤ log d for every state ρ on a d-dimensional Hilbert
space. Moreover H (ρ) = 0 if ρ is pure and H (ρ) = log d if the state is maximally
mixed.
Suppose that the state ρ AB of a composite system is given by the pure entangled
state in (2.26), already written in its Schmidt decomposition. Then, the von Neumann
entropy of the composite system is: H (AB) ρ = 0. From the Schmidt decomposition
(c.f. Sect. 2.4) we learned that the eigenvalues of ρ A and ρ B are equal and given
by the squares of the Schmidt coefficients. This leads to: H (A) ρ = H (B) ρ = 1.
2 Elements of Quantum Information Theory
h( p) := − p log p − (1 − p) log(1 − p)
(2.46)
Given two random variables X and Y jointly distributed according to { p(x, y)}, the
joint Shannon entropy reads:
H (XY ) = −p(x, y)
x,y
p(x, y),
(2.47)
while the conditional entropy is defined as:
H (X |Y ) = H (XY ) − H (Y ).
(2.48)
The conditional entropy of X given Y quantifies how uncertain we are about X , given
that we learned the value of Y . Finally, the mutual information H (X : Y ) measures
the amount of information we gain on X by observing the value of Y . This is given
by the total amount of information of X , H (X ), minus the uncertainty that we still
have on X after learning Y , i.e. H (X |Y ). Thus we have:
H (X : Y ) = H (X ) − H (X |Y ) = H (X ) + H (Y ) − H (XY ).
(2.49)
Of the many properties satisfied by the above-defined entropies, we highlight in
particular that: H (X |Y ) = H (XY ) − H (Y ) ≥ 0. We could intuitively expect this,
since the uncertainty on both random variables X and Y must be greater than the
uncertainty on Y . Conversely, this does not hold in general for quantum states, whose
uncertainty is quantified by the von Neumann entropy.
Definition 2.7 (von Neumann entropy) The von Neumann entropy of a quantum
state ρ, with eigenvalues {λ i }, is defined as:
H (ρ) = − Tr[ρ log ρ] = −
i
λ i log λ i .
(2.50)
Often, the von Neumann entropy of a system A in state ρ is indicated as: H (A) ρ .
One can interpret the von Neumann entropy of ρ as the Shannon entropy of the
probability distribution defined by its eigenvalues, hence we use the same symbol.
For this analogy, the previous definitions of joint entropy (2.47), conditional entropy
(2.48) and mutual information (2.49) can be extended to the von Neumann entropy.
We observe that 0 ≤ H (ρ) ≤ log d for every state ρ on a d-dimensional Hilbert
space. Moreover H (ρ) = 0 if ρ is pure and H (ρ) = log d if the state is maximally
mixed.
Suppose that the state ρ AB of a composite system is given by the pure entangled
state in (2.26), already written in its Schmidt decomposition. Then, the von Neumann
entropy of the composite system is: H (AB) ρ = 0. From the Schmidt decomposition
(c.f. Sect. 2.4) we learned that the eigenvalues of ρ A and ρ B are equal and given
by the squares of the Schmidt coefficients. This leads to: H (A) ρ = H (B) ρ = 1.