Chapter 10
Spinglass and Neural Networks
Abstract To begin with, neural networks are based on neural circuits formed by
neurons in human brains. One of the important mechanisms of the brain is memory.
The Hopfield model, which explains the mechanism of memory in terms of physics,
is a bridge between physics and neural networks. In this chapter, we explain the
Hopfield model and investigate the relationship between machine learning and spin
glass, which is still a rich subject in condensed matter physics.
The similarity between the neural network and the physical system cannot be
described without the Hopfield model [120]. This model explains the mechanism of
brain memory by a system in which many spins are coupled. 1 Let us consider that
many particles with spins gather and there are various interactions among the spins.
There appear many metastable states at low energy, and they degenerate. Such a
system is called spinglass. It is called spinglass because glass can take many states
instead of atoms being lined up in order like a solid. Such a classical spinglass
system can be considered as a kind of a neural network.
Linking neural networks to our familiar physical systems is one of the subjects of
this book. It is interesting to see that the spinglass system, which plays a leading role
in statistical physics, is related to neural networks, and it is one major intersection
that links physics and machine learning. Hopfield has modeled human memory
(information storage) using some collective dynamics of spinglasses, as we will
see below. This gives the idea that long-term memory can be considered as an
attractor of dynamical systems. Historically, from this perspective, the chaos of
brains, that is, the understanding of the brain as a dynamical system, has evolved.
In this chapter, we will return to the idea of Hopfield and look at the relationship
between spinglasses and neural networks.
1 S. Amari published a similar model ten years earlier [121], and the Hopfield’s model is also known
as the Amari–Hopfield model.
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021
A. Tanaka et al., Deep Learning and Physics, Mathematical Physics Studies,
https://doi.org/10.1007/978-981-33-6108-9_10
157
Spinglass and Neural Networks
Abstract To begin with, neural networks are based on neural circuits formed by
neurons in human brains. One of the important mechanisms of the brain is memory.
The Hopfield model, which explains the mechanism of memory in terms of physics,
is a bridge between physics and neural networks. In this chapter, we explain the
Hopfield model and investigate the relationship between machine learning and spin
glass, which is still a rich subject in condensed matter physics.
The similarity between the neural network and the physical system cannot be
described without the Hopfield model [120]. This model explains the mechanism of
brain memory by a system in which many spins are coupled. 1 Let us consider that
many particles with spins gather and there are various interactions among the spins.
There appear many metastable states at low energy, and they degenerate. Such a
system is called spinglass. It is called spinglass because glass can take many states
instead of atoms being lined up in order like a solid. Such a classical spinglass
system can be considered as a kind of a neural network.
Linking neural networks to our familiar physical systems is one of the subjects of
this book. It is interesting to see that the spinglass system, which plays a leading role
in statistical physics, is related to neural networks, and it is one major intersection
that links physics and machine learning. Hopfield has modeled human memory
(information storage) using some collective dynamics of spinglasses, as we will
see below. This gives the idea that long-term memory can be considered as an
attractor of dynamical systems. Historically, from this perspective, the chaos of
brains, that is, the understanding of the brain as a dynamical system, has evolved.
In this chapter, we will return to the idea of Hopfield and look at the relationship
between spinglasses and neural networks.
1 S. Amari published a similar model ten years earlier [121], and the Hopfield’s model is also known
as the Amari–Hopfield model.
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021
A. Tanaka et al., Deep Learning and Physics, Mathematical Physics Studies,
https://doi.org/10.1007/978-981-33-6108-9_10
157