178
F. Nielsen
∇
2 G Mult (η) =
1
1 −
i η i
1 M + diag
1
η 1
, . . . ,
1
η D
=
1
1−
i ηi
ifi = j,
1
1−
i ηi
+
1
ηi ifi = j
, (7.110)
where 1 M denotes the D × D-matrix with all entries equal to 1, and diag(x 1 , . . . , x D )
the diagonal matrix with diagonal elements x 1 , . . . , x D .
The dual Bregman divergences are
B F Mult (θ 1 : θ 2 ) = KL
∗
( p λ(θ 1 ) : p λ(θ 2 ) ) = KL( p λ(θ 2 ) : p λ(θ 1 ) ),
(7.111)
B G Mult (η 1 : η 2 ) = KL( p λ(η 1 ) : p λ(η 2 ) ),
(7.112)
where KL is the discrete Kullback–Leibler divergence and KL
∗ the reverse Kullback–
Leibler divergence. We convert the expectation parameter η to the original parameter
λ as follows:
λ(η) =
λ i = η i ,
fori ∈ {1, . . . , D}
λ d = 1 −
D
i=1 η i .
(7.113)
In general, the Bregman divergence induced by log-normalizer of an exponential
family amounts to a reverse Kullback–Leibler divergence [19]. Notice that the nonseparable log-normalizer of a multinomial family yields a nice example of a nonseparable Bregman divergence. The multinoulli manifold is called the Bernoulli
manifold when D = 1 (i.e., d = 2), and the trinoulli manifold when D = 2 (i.e.,
d = 3).
7.3 Geodesic ∇-Triangles with One, Two, or Three Interior
Right Angles
7.3.1 Geodesic ∇-Triangles with One Interior Right Angle
To build a geodesic ∇-triangle with a right angle, fix two points p and q (i.e., the
first two triangle vertices), and consider the location of the third triangle vertex point
r such that γ qr ⊥ q γ qp (i.e., ˙
γ qr (0) ⊥ q ˙
γ qp (0)). We end up with the following linear
equation which defines a θ -flat H
θ
q :
H
θ
q : θ
r ∇
2 F(θ q )θ pq = θ
q ∇
2 F(θ q )θ pq .
(7.114)
By restricting the solution r ∈ H
θ
q to the manifold M, we get:
Proposition 1 The locii of points r of a ∇-triangle that form a right angle at q is
H
θ
q ∩ M.
Figure 7.11 displays that a ∇-triangle with one right angle α q (r, p) =
π
2
in the
Itakura–Saito manifold: θ( p) = (1.2885253880864789, 3.4136709176658546),
F. Nielsen
∇
2 G Mult (η) =
1
1 −
i η i
1 M + diag
1
η 1
, . . . ,
1
η D
=
1
1−
i ηi
ifi = j,
1
1−
i ηi
+
1
ηi ifi = j
, (7.110)
where 1 M denotes the D × D-matrix with all entries equal to 1, and diag(x 1 , . . . , x D )
the diagonal matrix with diagonal elements x 1 , . . . , x D .
The dual Bregman divergences are
B F Mult (θ 1 : θ 2 ) = KL
∗
( p λ(θ 1 ) : p λ(θ 2 ) ) = KL( p λ(θ 2 ) : p λ(θ 1 ) ),
(7.111)
B G Mult (η 1 : η 2 ) = KL( p λ(η 1 ) : p λ(η 2 ) ),
(7.112)
where KL is the discrete Kullback–Leibler divergence and KL
∗ the reverse Kullback–
Leibler divergence. We convert the expectation parameter η to the original parameter
λ as follows:
λ(η) =
λ i = η i ,
fori ∈ {1, . . . , D}
λ d = 1 −
D
i=1 η i .
(7.113)
In general, the Bregman divergence induced by log-normalizer of an exponential
family amounts to a reverse Kullback–Leibler divergence [19]. Notice that the nonseparable log-normalizer of a multinomial family yields a nice example of a nonseparable Bregman divergence. The multinoulli manifold is called the Bernoulli
manifold when D = 1 (i.e., d = 2), and the trinoulli manifold when D = 2 (i.e.,
d = 3).
7.3 Geodesic ∇-Triangles with One, Two, or Three Interior
Right Angles
7.3.1 Geodesic ∇-Triangles with One Interior Right Angle
To build a geodesic ∇-triangle with a right angle, fix two points p and q (i.e., the
first two triangle vertices), and consider the location of the third triangle vertex point
r such that γ qr ⊥ q γ qp (i.e., ˙
γ qr (0) ⊥ q ˙
γ qp (0)). We end up with the following linear
equation which defines a θ -flat H
θ
q :
H
θ
q : θ
r ∇
2 F(θ q )θ pq = θ
q ∇
2 F(θ q )θ pq .
(7.114)
By restricting the solution r ∈ H
θ
q to the manifold M, we get:
Proposition 1 The locii of points r of a ∇-triangle that form a right angle at q is
H
θ
q ∩ M.
Figure 7.11 displays that a ∇-triangle with one right angle α q (r, p) =
π
2
in the
Itakura–Saito manifold: θ( p) = (1.2885253880864789, 3.4136709176658546),