106
6 Unsupervised Deep Learning
and the resultant effective Hamiltonian, 3
H
eff
J (x) = log Z J − log
h
e
−H J (x,h) .
(6.12)
And we define the model as
Q J (x) = e
−H eff
J (x) .
(6.13)
The model whose interaction is restricted in this way is called restricted Boltzmann
machine.
Learning in a restricted Boltzmann machine
Substituting (6.12) for H J (x) in the expression (6.10), we find
∂ J K(J )
= =∂ J H
eff
J (x) x∼P − −∂ J H
eff
J (y) y∼Q J
= =∂ J
log Z J − log
h
e
−H J (x,h)
x∼P − −∂ J
log Z J − log
h
e
−H J (y,h)
y∼Q J
=
h n
e −H J (x,h n ) ∂ J H J (x, h n )
h d
e −H J (x,h d )
x∼P
−
h n
e −H J (y,h n ) ∂ J H J (y, h n )
h d
e −H J (y,h d )
y∼Q J
=
h n
P J (h n |x)∂ J H J (x, h n )
x∼P
−
h n
P J (h n |y)∂ J H J (y, h n )
y∼Q J
.
(6.14)
Here
P J (h n |x) =
e −H J (x,h n )
h d
e −H J (x,h d )
(6.15)
is the conditional probability described in the column in Chap. 2. The inverse
conditional probability
P J (x n |h) =
e −H J (x n ,h)
x d
e −H J (x d ,h)
(6.16)
will also be needed below.
3 The effective Hamiltonian is defined to satisfy the following equation: exp[−H eff
J (x)] =
h exp[−H J (x, h)]/Z J .