7.1 Inverse Problems and Learning
131
network. In other words, the coefficients of the unknown function correspond to the
weights of the neural network.
If the form of the function f (x) is known to a certain extent, the term “modeling”
is often used, rather than machine learning. Due to physical requirements, the
shape of f (x) is restricted. For example, if we know the basic behavior such as
a heat diffusion system and wave system, and if external field fluctuations and other
interactions are added to it, the basic form of f (x) is determined, and one will write
the model in a perturbative way with small effects. The additional effect adds a
new term to the function, so its coefficient is unknown. This unknown coefficient is
determined from the input data and the output data. For example, if it is known that
f (x) is a linear function of x, a linear regression suffices.
On the other hand, in the case of deep learning, since the function form is
versatile, 4 we deal with a very wide class of functions without specifying the model.
And by the training of the machine, we gradually limit the model, that is, the
shape of the function. Therefore, machine learning and deep learning are the inverse
problems of the latter type among the above two meanings, and can be said to be
those which do not assume a model (function form) as its unknown function.
In this way, the nature of the inverse problem in the map (7.1) greatly depends
on which part is considered unknown. Learning is considered to be a kind of inverse
problems as described above. But when applying it to a physical system or exploring
an analogy with a physical system, one needs to be clear about what one is trying to
solve and which part is unknown.
The general properties of a problem called the “inverse problem” include the
following:
• Knowing objects that cannot be measured directly
• Infering the cause from the results
• Determining physical laws and governing equations
• Determining physical constants
An inverse problem is one of the above, requiring a way to solve it in the reverse
direction. Of these, things like “determining the laws of physics” are the most
important things in physics, and it is no exaggeration to say that all physics
innovations are inverse problems. Johannes Kepler discovered his third law of
planetary motion from Tycho Brahe’s precise planetary observation data. Max
Planck discovered the radiation formula. The underlying regularity in the data
sublimated to the law, which is the inverse problem. The importance of it is obvious.
Machine learning is a technique that has the potential to apply to all four of these
properties. Of course, the underlying mechanism, such as generalization, is not well
understood yet, but instead of relying solely on the physical sense of those called
“genius physicists,” machine learning provides a powerful and general method for
inverse problems.
4 Refer to Chap. 3 for the universal approximation theorem.
131
network. In other words, the coefficients of the unknown function correspond to the
weights of the neural network.
If the form of the function f (x) is known to a certain extent, the term “modeling”
is often used, rather than machine learning. Due to physical requirements, the
shape of f (x) is restricted. For example, if we know the basic behavior such as
a heat diffusion system and wave system, and if external field fluctuations and other
interactions are added to it, the basic form of f (x) is determined, and one will write
the model in a perturbative way with small effects. The additional effect adds a
new term to the function, so its coefficient is unknown. This unknown coefficient is
determined from the input data and the output data. For example, if it is known that
f (x) is a linear function of x, a linear regression suffices.
On the other hand, in the case of deep learning, since the function form is
versatile, 4 we deal with a very wide class of functions without specifying the model.
And by the training of the machine, we gradually limit the model, that is, the
shape of the function. Therefore, machine learning and deep learning are the inverse
problems of the latter type among the above two meanings, and can be said to be
those which do not assume a model (function form) as its unknown function.
In this way, the nature of the inverse problem in the map (7.1) greatly depends
on which part is considered unknown. Learning is considered to be a kind of inverse
problems as described above. But when applying it to a physical system or exploring
an analogy with a physical system, one needs to be clear about what one is trying to
solve and which part is unknown.
The general properties of a problem called the “inverse problem” include the
following:
• Knowing objects that cannot be measured directly
• Infering the cause from the results
• Determining physical laws and governing equations
• Determining physical constants
An inverse problem is one of the above, requiring a way to solve it in the reverse
direction. Of these, things like “determining the laws of physics” are the most
important things in physics, and it is no exaggeration to say that all physics
innovations are inverse problems. Johannes Kepler discovered his third law of
planetary motion from Tycho Brahe’s precise planetary observation data. Max
Planck discovered the radiation formula. The underlying regularity in the data
sublimated to the law, which is the inverse problem. The importance of it is obvious.
Machine learning is a technique that has the potential to apply to all four of these
properties. Of course, the underlying mechanism, such as generalization, is not well
understood yet, but instead of relying solely on the physical sense of those called
“genius physicists,” machine learning provides a powerful and general method for
inverse problems.
4 Refer to Chap. 3 for the universal approximation theorem.