10.3 Synchronization and Layering
163
The first term on the right-hand side is the attenuation term. The second term on
the right-hand side is a reinforcement term that depends on the state of both ends
connected by the synapse. When the memory pattern {S i = a
(m)
i } is input from
outside for a long period of time, J will be updated according to this kind of
equation. Since the equation is written to converge exponentially to (10.11), the
shape of the reinforcement term provides the appropriate weight.
10.3 Synchronization and Layering
Updating only randomly selected spins according to the rule (10.1) is not similar to
that of a multilayer deep neural network. However, modifying the Hopfield model
such that it updates all spins at the same time makes it possible to incorporate a
hierarchical structure.
Since we update the state of all spins at once, we shall label the number of steps
in that synchronized update as n:
S i (n) → S i (n + 1) ≡ sgn
N+1
i=1
J ij S j (n)
.
(10.19)
Then, as a Lyapunov function, it is known that the following choice works well:
E(n) ≡ −
1
2
ij
J ij S i (n + 1)S j (n) .
(10.20)
In fact, with a quantity that shows how E(n) changes with updates,
≡ E(n + 1) − E(n) ,
(10.21)
we can show that it is always negative or zero, as follows. First, let us massage
E(n) = −
1
2
ij
J ij (S i (n + 2) − S i (n)) S j (n + 1)
= −
1
2
i
(S i (n + 2) − S i (n))
j
J ij S j (n + 1)
= −
1
2
i
(S i (n + 2) − S i (n))
S i (n + 2)
|
j J ij S j (n + 1)|
.
(10.22)
163
The first term on the right-hand side is the attenuation term. The second term on
the right-hand side is a reinforcement term that depends on the state of both ends
connected by the synapse. When the memory pattern {S i = a
(m)
i } is input from
outside for a long period of time, J will be updated according to this kind of
equation. Since the equation is written to converge exponentially to (10.11), the
shape of the reinforcement term provides the appropriate weight.
10.3 Synchronization and Layering
Updating only randomly selected spins according to the rule (10.1) is not similar to
that of a multilayer deep neural network. However, modifying the Hopfield model
such that it updates all spins at the same time makes it possible to incorporate a
hierarchical structure.
Since we update the state of all spins at once, we shall label the number of steps
in that synchronized update as n:
S i (n) → S i (n + 1) ≡ sgn
N+1
i=1
J ij S j (n)
.
(10.19)
Then, as a Lyapunov function, it is known that the following choice works well:
E(n) ≡ −
1
2
ij
J ij S i (n + 1)S j (n) .
(10.20)
In fact, with a quantity that shows how E(n) changes with updates,
≡ E(n + 1) − E(n) ,
(10.21)
we can show that it is always negative or zero, as follows. First, let us massage
E(n) = −
1
2
ij
J ij (S i (n + 2) − S i (n)) S j (n + 1)
= −
1
2
i
(S i (n + 2) − S i (n))
j
J ij S j (n + 1)
= −
1
2
i
(S i (n + 2) − S i (n))
S i (n + 2)
|
j J ij S j (n + 1)|
.
(10.22)