7 On Geodesic Triangles with Right Angles in a Dually Flat Space
177
= exp
⎛
⎜
⎜
⎜
⎜
⎝
d−1
i=1
x i log λ i + (1 −
d−1
i=1
x i )
x d
log λ d
⎞
⎟
⎟
⎟
⎟
⎠
, (7.102)
= exp
d−1
i=1
x i θ i − F(θ )
,
(7.103)
with the natural parameter θ i = log
λ i
λ d
for i ∈ {1, . . . , d − 1}. The multinoulli distribution is an exponential family of order D = d − 1. The natural parameter space is
R
D
= R
d−1 . We can convert back the natural parameter θ to the original parameters
λ as follows:
λ(θ ) =
λ i =
exp(θ i )
1+
D
i=1 exp(θ i )
, fori ∈ {1, . . . , D}
λ d =
1
1+
D
i=1 exp(θ i )
.
(7.104)
The log-normalizer is
F Mult (θ ) = − log λ d = log
1 +
D
i=1
exp(θ i )
.
(7.105)
The gradient of the log-normalizer is
η(θ) = ∇ F Mult (θ ) =
exp(θ i )
1 +
D
i=1 exp(θ i )
i
,
(7.106)
and the reciprocal gradient is
θ(η) = ∇G Mult (η) =
log
η i
1 −
D
i=1
η i
i
.
(7.107)
The convex conjugate of F is
G Mult (η) = η
θ(η) − F(θ (η)) =
i
η i log η i +
1 −
i
η i
log
1 −
i
η i
.
(7.108)
It follows that the Riemannian metric and dual Riemannian metric are Hessians of
the potential functions F Mult (θ ) and G Mult (η), respectively:
∇
2 F Mult (θ)
i j
=
⎧
⎨
⎩
−
exp(θi +θ j )
(1+
i exp(θi ))
2
ifi = j,
exp(θi )
(1+
i exp(θi ))
−
exp(2θi )
(1+
i exp(θi ))
2 ifi = j
(7.109)