132
7 Inverse Problems in Physics
7.2 Regularization in Inverse Problems
Now, to look more closely at the relationship between machine learning and
inverse problems, consider the case of a linear function, instead of the nonlinear
function (7.1), where the nature of the inverse problem is clearer. As we will see
below, there are ways to solve inverse problems well for linear function problems.
Along the way, we introduce the important concept called regularization.
First, consider the following linear transformation:
y i =
m
k=1
J ik x k .
(7.2)
Here, the input or the initial value is the vector x k . Let its dimension be m. If the
dimension of the output y i is n, J is a matrix of the size n × m.
In this linear equation (7.2), consider the usual (the first one in the above sense)
inverse problem of finding x k for a given y i . In the case of a square matrix n = m,
the story is simple. If the inverse matrix J −1 of the matrix J is obtained, then x k is
obtained as follows:
x k =
m
i=1
J
−1
ki
y i .
(7.3)
While this inverse problem seems to work, there are actually two difficulties that
can arise:
• First, when the determinant of the matrix J is very close to zero numerically, the
difficulty is that x k changes significantly even for slightly different values of y i .
• Second, if the number of data y i is not enough, it is difficult to determine x k even
if there is an inverse matrix. This can be considered equivalent to the case n < m.
These issues are related in terms of data errors. When data is actually handled,
it always comes with fluctuations and errors of the measurements. Taking these
factors into account, solving the inverse problem can have such difficulties, even for
a simple linear problem.
Generally, well-posed problems defined by J. S. Hadamard refers to a problem
that satisfies the following three properties:
(1) Existence of a solution
(2) Uniqueness of the solution
(3) Stability of the solution (the solution changes infinitesimally when the initial
condition changes infinitesimally)
Problems where any of these are not met are called ill-posed problems. The inverse
problem which we described as the first example in the above is an ill-posed
problem.
7 Inverse Problems in Physics
7.2 Regularization in Inverse Problems
Now, to look more closely at the relationship between machine learning and
inverse problems, consider the case of a linear function, instead of the nonlinear
function (7.1), where the nature of the inverse problem is clearer. As we will see
below, there are ways to solve inverse problems well for linear function problems.
Along the way, we introduce the important concept called regularization.
First, consider the following linear transformation:
y i =
m
k=1
J ik x k .
(7.2)
Here, the input or the initial value is the vector x k . Let its dimension be m. If the
dimension of the output y i is n, J is a matrix of the size n × m.
In this linear equation (7.2), consider the usual (the first one in the above sense)
inverse problem of finding x k for a given y i . In the case of a square matrix n = m,
the story is simple. If the inverse matrix J −1 of the matrix J is obtained, then x k is
obtained as follows:
x k =
m
i=1
J
−1
ki
y i .
(7.3)
While this inverse problem seems to work, there are actually two difficulties that
can arise:
• First, when the determinant of the matrix J is very close to zero numerically, the
difficulty is that x k changes significantly even for slightly different values of y i .
• Second, if the number of data y i is not enough, it is difficult to determine x k even
if there is an inverse matrix. This can be considered equivalent to the case n < m.
These issues are related in terms of data errors. When data is actually handled,
it always comes with fluctuations and errors of the measurements. Taking these
factors into account, solving the inverse problem can have such difficulties, even for
a simple linear problem.
Generally, well-posed problems defined by J. S. Hadamard refers to a problem
that satisfies the following three properties:
(1) Existence of a solution
(2) Uniqueness of the solution
(3) Stability of the solution (the solution changes infinitesimally when the initial
condition changes infinitesimally)
Problems where any of these are not met are called ill-posed problems. The inverse
problem which we described as the first example in the above is an ill-posed
problem.