34
2 Introduction to Machine Learning
We calculate the relative entropy up to the second order in dσ, dμ as
D KL (p||q) =
1
2
− log
σ 2
(σ + dσ ) 2 +
σ 2
(σ + dσ ) 2 − 1
+
1
(σ + dσ ) 2 (μ − μ − dμ)
2
=
1
2
log (1 +
dσ
σ
)
2 +
1
(1 +
dσ
σ ) 2
− 1
+
1
σ 2 (1 +
dσ
σ ) 2
dμ
2
≈
1
2
2
dσ
σ
−
dσ 2
σ 2 +
1 − 2
dσ
σ
+ 3
dσ 2
σ 2 − 1
+
dμ 2
σ 2
=
1
2
2
dσ 2
σ 2 +
dμ 2
σ 2
=
dσ 2 + d ˜
μ 2
σ 2
.
(2.55)
(In the last expression, we defined ˜
μ = μ/
√
2.) This is the metric of a hyperboloid,
and a part of the metric of the anti-de Sitter spacetime (AdS spacetime). In recent
years, the anti-de Sitter spacetime has played an important role in phenomena called
AdS/CFT correspondence 19 in string theory. It is interesting that Gaussian space
also has such a “hidden” anti-de Sitter spacetime. Anti-de-Sitter spacetime has the
property that the metric diverges at its infinite boundary (the distance is stretched
infinitely). In the current context it corresponds to σ ≈ 0. σ is the variance of the
Gaussian distribution, so the origin of this anomalous behavior is clear: the limit is
no longer a function in the usual sense, since the limit of bringing the variance of
the Gaussian distribution to zero corresponds to the Dirac delta function. Chapter 12
introduces research relating AdS/CFT correspondence to machine learning.
19 CFT (conformal field theory) is a special class of quantum field theory.