174
F. Nielsen
F
∗
(η) =
1
2
η
Q
−1
η = F Q −1 (η),
(7.79)
and it follows that
η(θ) = ∇ F Q (θ ) = Qθ, θ(η) = ∇ F
∗
Q (η) = Q
−1
η.
(7.80)
The dual Riemannian metrics are
g i j
= ∇ 2 F Q (θ) = Q,
g ∗i j
(7.81)
= ∇ 2 F ∗
Q (η) = Q −1 .
(7.82)
The dual geodesics γ pq and γ
∗
qp in a Mahalanobis manifold coincide. The dual Bregman divergences are squared Mahalanobis distances [12, 23]:
B F Q (θ 1 : θ 2 ) =
1
2
(θ 1 − θ 2 )
Q(θ 1 − θ 2 ),
(7.83)
and
B F
∗
Q
(η 1 : η 2 ) =
1
2
(η 1 − η 2 )
Q
−1
(η 1 − η 2 ).
(7.84)
We check that
B F
∗
Q
(η 2 : η 1 ) =
1
2
(Q(θ 2 − θ 1 ))
Q
−1 Q(θ 2 − θ 1 ) = B F Q (θ 1 : θ 2 ),
(7.85)
since Q
= Q. The squared Mahalanobis Bregman divergences are provably the only
symmetric Bregman divergences [5]. In particular, when Q = I , the identity matrix,
the dual Bregman divergences B F and B
∗
F coincide with half of the squared Euclidean
distance D E (θ 1 , θ 2 ) =
1
2
(θ 2 − θ 1 )
(θ 2 − θ 1 ). By using the Cholesky decomposition
of Q, i.e., Q = L L
where L is a lower triangular matrix with positive diagonal entries, we have B F Q (θ 1 : θ 2 ) = B F I (L
θ 1 : L
θ 2 ) = =L
(θ 1 − θ 2 )
2 , where I
denotes the identity matrix. Notice that the squared Mahalanobis divergences B F Q
are non-separable Bregman divergences whenever Q has non-zero off-diagonal elements.
7.2.4.2 The Kullback–Leibler Manifold
The extended Shannon negative entropy [16]
F KL (θ ) =
D
i=1
θ
i log θ
i
− θ
i
(7.86)