8.3 What the Neural Network Sees
145
Fig. 8.3 Left: θ ∗ for the case that the intermediate layer has 3 units. Right: Correlation between
each component of u and the magnetization. (Excerpt from [109])
The output is normalized and so can be regarded as a probability. θ and ϕ are the
parameters to be trained (weights). The results of the numerical experiments are
shown on the left of Fig. 8.3. Looking at the θ ∗ obtained in this case, we can see that
the phase transition is again detected.
By the way, what kind of information is hidden in the following intermediate
three-dimensional vector?
u =
⎛
⎝
u red
u green
u blue
⎞
⎠ = ϕ
∗ x .
(8.11)
The vector components are plotted for various spin configurations (the magnetization M on the horizontal axis and u on the vertical axis). See the right panel of
Fig. 8.3. As you can see, there exists a clear correlation, so we can conclude that the
order parameter of the phase transition has been learned at the middle layer.
Once you deepen the network, you can actually confirm that even the information
about the internal energy is directly embedded. This suggests a theoretical connection between neural networks and statistical systems. Interested readers should refer
to the paper [109], and a related reference [110].
145
Fig. 8.3 Left: θ ∗ for the case that the intermediate layer has 3 units. Right: Correlation between
each component of u and the magnetization. (Excerpt from [109])
The output is normalized and so can be regarded as a probability. θ and ϕ are the
parameters to be trained (weights). The results of the numerical experiments are
shown on the left of Fig. 8.3. Looking at the θ ∗ obtained in this case, we can see that
the phase transition is again detected.
By the way, what kind of information is hidden in the following intermediate
three-dimensional vector?
u =
⎛
⎝
u red
u green
u blue
⎞
⎠ = ϕ
∗ x .
(8.11)
The vector components are plotted for various spin configurations (the magnetization M on the horizontal axis and u on the vertical axis). See the right panel of
Fig. 8.3. As you can see, there exists a clear correlation, so we can conclude that the
order parameter of the phase transition has been learned at the middle layer.
Once you deepen the network, you can actually confirm that even the information
about the internal energy is directly embedded. This suggests a theoretical connection between neural networks and statistical systems. Interested readers should refer
to the paper [109], and a related reference [110].