4
1 Forewords: Machine Learning and Physics
The “information” here is the information we already have, and the amount of
information (1.4) is the information obtained from the event.
1.2 Physics and Information Theory
There are many important concepts in physics, but no physicist would oppose
that one of the most important is entropy. Entropy is an essential concept in the
development of thermodynamics, and is expressed as S = k B log W , where W
is the number of microscopic states, in statistical mechanics. The proportionality
coefficient k B is the Boltzmann constant and can be set to 1 if an appropriate
temperature unit is used. The entropy of the system in this case is
S = log W .
(1.9)
In physics, the relationship with information theory is revealed through this entropy:
That is because S information of (1.7) is exactly the same formula as (1.9). In this way,
various discussions on physical systems dealing with multiple degrees of freedom,
such as thermodynamics and statistical mechanics, have a companion informationtheoretic interpretation.
By the way, most of the interest in research in modern physics (e.g., particle
physics and condensed matter physics) is in many-body systems. This suggests
that information theory plays an important role at the forefront of modern physics
research. Here are two recent studies.
Black hole information loss problem
Research by J. Bekenstein and S. Hawking [6, 7] shows theoretically that black
holes have entropy and radiate their mass outward as heat. To explain the intrinsic
problem hidden in this radiation property of black holes, the following example is
instructive: Assume that there is a spring, and fix it while stretched. If this is thrown
into a black hole, the black hole will grow larger by the stored energy, and then
emit that energy as heat with the aforementioned thermal radiation. But here is the
problem. Work can be freely extracted from the energy of the spring before it is
thrown, but the second law of thermodynamics limits the energy efficiency that can
be extracted from the thermal radiation. In other words, even if there is only a single
state before the throwing (that is, the entropy is zero), the entropy increases by the
randomness after thermal radiation. An increase in entropy means that information
has been lost (see (1.8)). This is known as the information loss problem and is one
of the most important issues in modern physics for which no definitive solution has
yet been obtained. 3
3 For this, take a look at the column in Chap. 12.
1 Forewords: Machine Learning and Physics
The “information” here is the information we already have, and the amount of
information (1.4) is the information obtained from the event.
1.2 Physics and Information Theory
There are many important concepts in physics, but no physicist would oppose
that one of the most important is entropy. Entropy is an essential concept in the
development of thermodynamics, and is expressed as S = k B log W , where W
is the number of microscopic states, in statistical mechanics. The proportionality
coefficient k B is the Boltzmann constant and can be set to 1 if an appropriate
temperature unit is used. The entropy of the system in this case is
S = log W .
(1.9)
In physics, the relationship with information theory is revealed through this entropy:
That is because S information of (1.7) is exactly the same formula as (1.9). In this way,
various discussions on physical systems dealing with multiple degrees of freedom,
such as thermodynamics and statistical mechanics, have a companion informationtheoretic interpretation.
By the way, most of the interest in research in modern physics (e.g., particle
physics and condensed matter physics) is in many-body systems. This suggests
that information theory plays an important role at the forefront of modern physics
research. Here are two recent studies.
Black hole information loss problem
Research by J. Bekenstein and S. Hawking [6, 7] shows theoretically that black
holes have entropy and radiate their mass outward as heat. To explain the intrinsic
problem hidden in this radiation property of black holes, the following example is
instructive: Assume that there is a spring, and fix it while stretched. If this is thrown
into a black hole, the black hole will grow larger by the stored energy, and then
emit that energy as heat with the aforementioned thermal radiation. But here is the
problem. Work can be freely extracted from the energy of the spring before it is
thrown, but the second law of thermodynamics limits the energy efficiency that can
be extracted from the thermal radiation. In other words, even if there is only a single
state before the throwing (that is, the entropy is zero), the entropy increases by the
randomness after thermal radiation. An increase in entropy means that information
has been lost (see (1.8)). This is known as the information loss problem and is one
of the most important issues in modern physics for which no definitive solution has
yet been obtained. 3
3 For this, take a look at the column in Chap. 12.