11.2 Tensor Networks and Neural Networks
169
Fig. 11.2 Quantum
states (11.6) and (11.7)
represented by tensor
networks
the other hand, the lines mean “multiplying weights W ,” which is the role of a
tensor in a tensor network. Also, in the neural network, the units (circles) have the
meaning of the input and the output, and in the tensor network, circles are tensors
(the role of weighting). Therefore, the meaning of lines and circles is reversed in
tensor networks and neural networks.
11.2.2 Tensor Network Representation of Restricted Boltzmann
Machines
Now let us look at the relationship between the restricted Boltzmann machines
and the tensor networks [125]. In the neural network graph that gives a restricted
Boltzmann machine, we associate the following 2 × 2 matrix with each line
connecting a visible unit s a and a hidden unit h A ,
M
(aA)
ss ≡
1
1
1 exp[J aA ]
ss
,
(11.8)
and also associate the following 2 × 2 matrix at the connection of the unit and the
line,
(a)
ss ≡
1
0
0 exp[a a ]
ss
, ˜
(A)
ss ≡
1
0
0 exp[b A ]
ss
.
(11.9)
With this rule, all restricted Boltzmann machines can be interpreted as tensor
networks.
For example, consider the simplest restricted Boltzmann machine made of a
single visible unit, a single hidden unit and a single line connecting them (Fig. 11.3
left). According to the above rules,
˜
(a=1) M
(1,1)
(A=1)
=
1
e a 1
e b 1 e a 1 +b 1 +J 11
.
(11.10)
The elements of this matrix correspond to the weights of the restricted Boltzmann
machine to which (s, h) = (0, 0), (0, 1), (1, 0), (1, 1) is substituted. To make a