160
10 Spinglass and Neural Networks
of as physically equivalent to the energy of the system. The change in E under the
rule is
E = −
i,j
J ij S j i .
(10.9)
If i = (+1) − (−1) = 2, then
i,j J ij S j > 0, so the contribution is E < 0.
If i = (−1) − (+1) = −2, then
i,j J ij S j < 0, so again the contribution
is negative, namely E < 0. If i = 0, the contribution has no effect on
Therefore, we can show
≤ 0.
(10.10)
That is, E is invariant or monotonically decreasing and satisfies the properties of
any Lyapunov function.
The meaning that the function E plays the role of energy should be understood in
the broad sense that the system proceeds in the direction of decreasing energy. 5 In
this sense, the Hopfield model is similar to the potential energy part of many-body
spin systems. In fact, the Hopfield model was the historical origin of the Boltzmann
machine described in Chap. 6.
What is the spin configuration that minimizes or extremizes the potential
energy (10.9)? Considering a general J ij , we can see that a very large number
of configurations achieve a local minimum. Such a system is called a spinglass.
For example, if spins are arranged at equal intervals on a one-dimensional line,
and only adjacent spins have a bond of J i,i+1 = J (> 0), the lowest energy
state realized is ferromagnetic, that is, S i = +1 for all i’s. In this case, there is
no degeneration. On the other hand, for example, considering only three spins, if
J 12 = J 13 = −J 23 = J (> 0), the lowest energy state has six patterns, thus there
is a sixfold degeneracy. The three spins are in a state of three-way deadlock, and if
you try to lower any one of the interaction energies, some other will increase. The
system in which such a state is realized is called the system with “frustration.” If
you allow a general coupling as J whose value can be a positive or negative general
value, you will have a frustrated system.
In summary, in the Hopfield model, when the firing state of each neuron evolves
according to the rule (10.1), the state finally reaches various states which are quite
degenerate.
5 Energy is conserved if it has a time-translation invariant Hamiltonian. In that sense, the Lyapunov
function is not equivalent to energy. However, for example, if the Hamiltonian system is in
contact with an external heatbath and the temperature of the heatbath is low, the energy drops
monotonically in time.
10 Spinglass and Neural Networks
of as physically equivalent to the energy of the system. The change in E under the
rule is
E = −
i,j
J ij S j i .
(10.9)
If i = (+1) − (−1) = 2, then
i,j J ij S j > 0, so the contribution is E < 0.
If i = (−1) − (+1) = −2, then
i,j J ij S j < 0, so again the contribution
is negative, namely E < 0. If i = 0, the contribution has no effect on
Therefore, we can show
≤ 0.
(10.10)
That is, E is invariant or monotonically decreasing and satisfies the properties of
any Lyapunov function.
The meaning that the function E plays the role of energy should be understood in
the broad sense that the system proceeds in the direction of decreasing energy. 5 In
this sense, the Hopfield model is similar to the potential energy part of many-body
spin systems. In fact, the Hopfield model was the historical origin of the Boltzmann
machine described in Chap. 6.
What is the spin configuration that minimizes or extremizes the potential
energy (10.9)? Considering a general J ij , we can see that a very large number
of configurations achieve a local minimum. Such a system is called a spinglass.
For example, if spins are arranged at equal intervals on a one-dimensional line,
and only adjacent spins have a bond of J i,i+1 = J (> 0), the lowest energy
state realized is ferromagnetic, that is, S i = +1 for all i’s. In this case, there is
no degeneration. On the other hand, for example, considering only three spins, if
J 12 = J 13 = −J 23 = J (> 0), the lowest energy state has six patterns, thus there
is a sixfold degeneracy. The three spins are in a state of three-way deadlock, and if
you try to lower any one of the interaction energies, some other will increase. The
system in which such a state is realized is called the system with “frustration.” If
you allow a general coupling as J whose value can be a positive or negative general
value, you will have a frustrated system.
In summary, in the Hopfield model, when the firing state of each neuron evolves
according to the rule (10.1), the state finally reaches various states which are quite
degenerate.
5 Energy is conserved if it has a time-translation invariant Hamiltonian. In that sense, the Lyapunov
function is not equivalent to energy. However, for example, if the Hamiltonian system is in
contact with an external heatbath and the temperature of the heatbath is low, the energy drops
monotonically in time.