10
1 Forewords: Machine Learning and Physics
mechanics, especially the Ising model that deals with spin degrees of freedom. A
neural network is an artificial network that simulates the connection of neurons in
the brain, and the Hopfield model, which provides a mechanism of storing memory
as a physical system, can be said to be a type of Ising model. Focusing on these
points, in this book we describe various settings of neural networks from statistical
mechanics.
In Boltzmann machine learning and Bayesian statistics, it is necessary to generate
(or sample) (pseudo-)random numbers that follow some complicated probability
distribution. It is a popular method in numerical calculations in condensed matter
physics and elementary particle physics, so there is something in common here. 9
Sampling takes too long for complex models, so in recent years the Boltzmann
machine has been replaced by neural networks and is not attracting attention.
However, the academic significance may yet be recealed, so it would be a good
idea not only to follow the latest topics of deep learning, but also to return to the
beginning and rethink the Boltzmann machine.
Today, deep learning is simply a collection of various techniques that have
been found to work “empirically” when applying deep neural networks to various
machine learning schemes. However, the mind that senses any technique to work
“empirically” has something close to the so-called “physical sense”. 10 For example,
in recent years, it has been known that a model called ResNet, which includes a
“bypass” in a neural network, has a high capability. ResNet learns the “residual”
of the deep neural network, rather than learning directly the “features” that would
have been acquired by the ordinary deep neural networks. The “features” are the
quantities important for performing the desired task. In the ResNet, the residuals are
accumulated to express the features. This is expressed as a relationship “differential
= residual” and “integral = feature”, and is reminiscent of the equation of motion
and its solution.
In this sense, machine learning and physics seem to have “a certain” relationship.
To actually try the translation, we need to create a corresponding dictionary such as
Table 1.1. Returning to Fig. 1.1, combined with the many fragmentary relationships
listed above, machine learning and deep learning methodologies can be viewed and
constructed from a physical perspective.
With this in mind, this book is aimed at:
1. Understanding machine learning methods from a physics perspective
9 In fact, it is not unusual that people majoring in computational physics switch to the world of data
science.
10 As an example, most quantum field theories have not been mathematically justified, but
are empirically consistent with experimental results. They are formulated by a physical sense.
Calculated results of quantum electrodynamics, which is a branch of the quantum field theories,
contain a number of operations that are not mathematically justified, but they are more than
10 orders of magnitude consistent with experimental measurements. Of course, the performed
operations are consistent in the sense of theoretical physicists, but they have not been justified in a
mathematically rigorous way.
1 Forewords: Machine Learning and Physics
mechanics, especially the Ising model that deals with spin degrees of freedom. A
neural network is an artificial network that simulates the connection of neurons in
the brain, and the Hopfield model, which provides a mechanism of storing memory
as a physical system, can be said to be a type of Ising model. Focusing on these
points, in this book we describe various settings of neural networks from statistical
mechanics.
In Boltzmann machine learning and Bayesian statistics, it is necessary to generate
(or sample) (pseudo-)random numbers that follow some complicated probability
distribution. It is a popular method in numerical calculations in condensed matter
physics and elementary particle physics, so there is something in common here. 9
Sampling takes too long for complex models, so in recent years the Boltzmann
machine has been replaced by neural networks and is not attracting attention.
However, the academic significance may yet be recealed, so it would be a good
idea not only to follow the latest topics of deep learning, but also to return to the
beginning and rethink the Boltzmann machine.
Today, deep learning is simply a collection of various techniques that have
been found to work “empirically” when applying deep neural networks to various
machine learning schemes. However, the mind that senses any technique to work
“empirically” has something close to the so-called “physical sense”. 10 For example,
in recent years, it has been known that a model called ResNet, which includes a
“bypass” in a neural network, has a high capability. ResNet learns the “residual”
of the deep neural network, rather than learning directly the “features” that would
have been acquired by the ordinary deep neural networks. The “features” are the
quantities important for performing the desired task. In the ResNet, the residuals are
accumulated to express the features. This is expressed as a relationship “differential
= residual” and “integral = feature”, and is reminiscent of the equation of motion
and its solution.
In this sense, machine learning and physics seem to have “a certain” relationship.
To actually try the translation, we need to create a corresponding dictionary such as
Table 1.1. Returning to Fig. 1.1, combined with the many fragmentary relationships
listed above, machine learning and deep learning methodologies can be viewed and
constructed from a physical perspective.
With this in mind, this book is aimed at:
1. Understanding machine learning methods from a physics perspective
9 In fact, it is not unusual that people majoring in computational physics switch to the world of data
science.
10 As an example, most quantum field theories have not been mathematically justified, but
are empirically consistent with experimental results. They are formulated by a physical sense.
Calculated results of quantum electrodynamics, which is a branch of the quantum field theories,
contain a number of operations that are not mathematically justified, but they are more than
10 orders of magnitude consistent with experimental measurements. Of course, the performed
operations are consistent in the sense of theoretical physicists, but they have not been justified in a
mathematically rigorous way.