1.4 Machine Learning and Physics
9
We let the fairy to continue its work all night for five days. The result is:
It looks like a person’s face. 7 The “special fairy” had the goal of drawing this
picture by pressing/not-pressing buttons. This may seem like a trivial task, but here
we explain a bit more physically that it is not.
First, consider the random arrangement of 0 and 1 as the initial state. With this
we do not lose the generality of the problem. The fairy decides whether to replace
0/1 with 1/0 or leave it as it is at the time (x, y) (coordinates correspond to the time),
by operating the button. The fairy checks the state of the single bit at time (x, y) and
decides whether or not to invert it. Yes, the readers now know that the identity of
this “special fairy” is Maxwell’s demon. 8
From this point of view, the act of “drawing a picture” is to draw a meaningful
combination (low entropy) from among the myriad of possibilities (high entropy)
that can exist on the canvas. In other words, it is to make the information increase in
some way. You may have heard news that artificial intelligence has painted pictures.
Without fear of misunderstanding, we can say that it means a successful creation of
a Maxwell’s demon, in the current context.
The story of Maxwell’s demon above was a thought experiment that suggested
that machine learning and physics might be related, but in fact, can the subject of
this book, machine learning and physics, be connected by a thick pipe?
In fact, even in the field of machine learning, there are many problems that often
require “physical sense” and there are many research results inspired by it. For
example, in the early days of deep learning, a model (Boltzmann machine) that used
a stochastic neural network was used, and this model was inspired by statistical
7 This is a binary image of the sample data “Lenna image” [14]. It is used as a standard in research
fields such as image compression.
8 This analogy is a slightly modified version of the story in [15]. It would be interesting to discuss
the modern physics solution of the Maxwell’s demon problem in the context here, namely, where
does the entropy reduction used to learn some meaningful probability distribution in machine
learning come from?
9
We let the fairy to continue its work all night for five days. The result is:
It looks like a person’s face. 7 The “special fairy” had the goal of drawing this
picture by pressing/not-pressing buttons. This may seem like a trivial task, but here
we explain a bit more physically that it is not.
First, consider the random arrangement of 0 and 1 as the initial state. With this
we do not lose the generality of the problem. The fairy decides whether to replace
0/1 with 1/0 or leave it as it is at the time (x, y) (coordinates correspond to the time),
by operating the button. The fairy checks the state of the single bit at time (x, y) and
decides whether or not to invert it. Yes, the readers now know that the identity of
this “special fairy” is Maxwell’s demon. 8
From this point of view, the act of “drawing a picture” is to draw a meaningful
combination (low entropy) from among the myriad of possibilities (high entropy)
that can exist on the canvas. In other words, it is to make the information increase in
some way. You may have heard news that artificial intelligence has painted pictures.
Without fear of misunderstanding, we can say that it means a successful creation of
a Maxwell’s demon, in the current context.
The story of Maxwell’s demon above was a thought experiment that suggested
that machine learning and physics might be related, but in fact, can the subject of
this book, machine learning and physics, be connected by a thick pipe?
In fact, even in the field of machine learning, there are many problems that often
require “physical sense” and there are many research results inspired by it. For
example, in the early days of deep learning, a model (Boltzmann machine) that used
a stochastic neural network was used, and this model was inspired by statistical
7 This is a binary image of the sample data “Lenna image” [14]. It is used as a standard in research
fields such as image compression.
8 This analogy is a slightly modified version of the story in [15]. It would be interesting to discuss
the modern physics solution of the Maxwell’s demon problem in the context here, namely, where
does the entropy reduction used to learn some meaningful probability distribution in machine
learning come from?