114
6 Unsupervised Deep Learning
Can we extract any useful information from here? The point is to imagine a “perfect
generator” G perfect ,
Q G perfect (x) = P (x) .
(6.60)
Since in this proof G is assumed to have infinite expressive power (it can
approximate all possible functions), we may substitute G perfect into G. Since (6.59)
holds for any G, the inequality holds also for the “perfect generator”:
dx Q G ∗ (x)D
∗ (x) ≤
dx Q G perfect (x)D
∗ (x) =
dx P (x)D
∗ (x) .
(6.61)
The integral in this last expression is the same as the first term of (6.53) into which
D = D ∗ is substituted, so using this inequality we newly obtain
V D (G
∗ , D
∗ ) ≥
dx Q G ∗ (x)
D
∗ (x) + max(0, m − D
∗ (x))
.
(6.62)
Here, we can prove 11 that “almost everywhere” 12 the following holds:
D
∗ (x) ≤ m .
(6.66)
11 Let us prove it by reductio ad absurdum. The negation of the statement that almost everywhere
we have (6.66) is
S = {x|D
∗ (x) > m}contributes to the integral.
(6.63)
We define a new ˜
D(x) = min(m, D ∗ (x)) and substituting it into D of V D (G ∗ , D), we just put
D = ˜
D in (6.53),
V D (G
∗ , ˜
D) =
S∪S c
dx
P (x) ˜
D(x) + Q G ∗ (x) max(0, m − ˜
D(x))
=
S
dx
P (x) ˜
D(x)
=m +Q G ∗ (x) max(0, m − ˜
D(x))
=0 +
S c
dx
P (x) ˜
D(x)
D ∗ (x)
+Q G ∗ (x) max(0, m − ˜
D(x)
D ∗ (x)
)
<
S∪S c
dx
P (x)D
∗ (x) + Q G ∗ (x) max(0, m − D
∗ (x))
= V D (G
∗ , D
∗ ) .
(6.64)
This inequality contradicts the Nash equilibrium definition (6.40), where D ∗ gives the minimum
value of V D (G ∗ , D) for D.
12 This is equivalent to
1 D ∗ (x)>m dx = 0 .
(6.65)
In other words, the support set S = {x|D ∗ (x) > m} does not contribute to the integral.