14
I Physical View of Deep Learning
how to handle data in probability theory, and describe “generalization” and its
importance in learning.
Chapter 3: Basics of Neural Networks Next, in this chapter, we derive neural
networks from the viewpoint of physical models. A neural network is a nonlinear
function that maps an input to an output, and giving the network is equivalent
to giving a function called an error function in the case of supervised learning.
By considering the output as dynamical degrees of freedom and the input as an
external field, various neural networks and their deepened versions are born from
simple Hamiltonians. 1 Training (learning) is a procedure for reducing the value
of the error function, and we will learn the specific method of backpropagation
using the bra-ket notation popular in quantum mechanics. And we will look
at how the “universal approximation theorem” works, which is why neural
networks can express connections between various types of data.
Chapter 4: Advanced Neural Networks In this chapter, we explain the structure
of the two types of neural networks that have been the mainstays of deep
learning in recent years, following the words of physics in the previous chapter.
A convolutional neural network has a structure that emphasizes the spatial
proximity in input data. Also, recurrent neural networks have a structure to learn
input data in time series. You will learn how to provide a network structure that
respects the characteristics of data.
Chapter 5: Sampling In the situation where the training is performed, it is assumed
that the input data is given by a probability distribution. It is often necessary to
calculate the expectation value of the function of various input values given by
the probability distribution. In this chapter, we will look at the method and the
necessity of “sampling,” which is the method of performing the calculation of the
expectation value. The frequently used concepts in statistical mechanics, such as
the law of large numbers, the central limit theorem, the Markov chain Monte
Carlo method, the principle of detailed balance, the Metropolis method, and the
heatbath method, are also used in machine learning. Familiarity with common
concepts in physics and machine learning can lead to an understanding of both.
Chapter 6: Unsupervised Deep Learning At the end of Part I we will explain
Boltzmann machines and generative adversarial networks (GANs). Both models
are not the “find an answer” network given in Chap. 3, but rather the network
itself giving the probability distribution of the input. Boltzmann machines
have historically been the cornerstone of neural networks and are given by
the Hamiltonian statistical mechanics of multi-particle spin systems. It is an
important bridge between machine learning and physics. Generative adversarial
networks are also one of the important topics in deep learning in recent years,
and we try to provide an explanation of them from a physical point of view.
1 For the basic concepts of physics used in this book, statistical mechanics and quantum mechanics,
see the columns at the end of chapters.
I Physical View of Deep Learning
how to handle data in probability theory, and describe “generalization” and its
importance in learning.
Chapter 3: Basics of Neural Networks Next, in this chapter, we derive neural
networks from the viewpoint of physical models. A neural network is a nonlinear
function that maps an input to an output, and giving the network is equivalent
to giving a function called an error function in the case of supervised learning.
By considering the output as dynamical degrees of freedom and the input as an
external field, various neural networks and their deepened versions are born from
simple Hamiltonians. 1 Training (learning) is a procedure for reducing the value
of the error function, and we will learn the specific method of backpropagation
using the bra-ket notation popular in quantum mechanics. And we will look
at how the “universal approximation theorem” works, which is why neural
networks can express connections between various types of data.
Chapter 4: Advanced Neural Networks In this chapter, we explain the structure
of the two types of neural networks that have been the mainstays of deep
learning in recent years, following the words of physics in the previous chapter.
A convolutional neural network has a structure that emphasizes the spatial
proximity in input data. Also, recurrent neural networks have a structure to learn
input data in time series. You will learn how to provide a network structure that
respects the characteristics of data.
Chapter 5: Sampling In the situation where the training is performed, it is assumed
that the input data is given by a probability distribution. It is often necessary to
calculate the expectation value of the function of various input values given by
the probability distribution. In this chapter, we will look at the method and the
necessity of “sampling,” which is the method of performing the calculation of the
expectation value. The frequently used concepts in statistical mechanics, such as
the law of large numbers, the central limit theorem, the Markov chain Monte
Carlo method, the principle of detailed balance, the Metropolis method, and the
heatbath method, are also used in machine learning. Familiarity with common
concepts in physics and machine learning can lead to an understanding of both.
Chapter 6: Unsupervised Deep Learning At the end of Part I we will explain
Boltzmann machines and generative adversarial networks (GANs). Both models
are not the “find an answer” network given in Chap. 3, but rather the network
itself giving the probability distribution of the input. Boltzmann machines
have historically been the cornerstone of neural networks and are given by
the Hamiltonian statistical mechanics of multi-particle spin systems. It is an
important bridge between machine learning and physics. Generative adversarial
networks are also one of the important topics in deep learning in recent years,
and we try to provide an explanation of them from a physical point of view.
1 For the basic concepts of physics used in this book, statistical mechanics and quantum mechanics,
see the columns at the end of chapters.