6.6 Nonlinear Modeling Method Based on DBN
231
Fig. 6.24 Diagram of RBM
structure
By constructing an effective energy function, the minimum value of the energy
function can be determined. According to the energy function, the probabilities of
visible nodes and hidden nodes can be defined as
p(v|h) =
e
−E(v,h)
h e −E(v,h) p(h|v) =
e
−E(v,h)
v e −E(v,h)
(6.25)
By setting the parameters W, b, and c to be θ , the likelihood estimation of the
joint probability of v and h can be obtained as in Eq. (6.26).
P θ (v, h) =
1
Z (θ )
e
−E(v,h|θ)
(6.26)
In Eq. (6.26), Z (θ ) is the normalization factor, which is also referred to as the
partition function. Combined with Eq. (6.27), the above equation can be written as
P θ (v, h) =
1
Z (θ )
exp(−
n
i=1
m
j=1
ω i j h i v j −
m
j=1
b j v j −
n
i=1
c i h i )
(6.27)
The likelihood function p θ (v, h) of the observation data is maximized to acquire
parameters of RBMs. However, when the sample is large, traversing all possible
values of v and h will make the computational complexity high. To reduce computational complexity, the input of RBMs is fitted by Gibbs sampling. Contrastive
divergence (CD) is a simplified algorithm for Gibbs sampling. The advantage of the
CD is that the initial training sample requires only a small number of sampling steps.