114 Beam-based Correction and Optimization for Accelerators
in steps while observing the ∆K values in the fitting solution and the resulting χ
2 . A set of weight factors can be accepted if the fitting solution brings
χ
2 down to the same level as the case without constraints (or the case with
very weak constraints if no solution can be found without constraints) with
reasonable values of
∆K
K . Unless there is a justifiable reason, the fitted
∆K
K
for a quadrupole parameter (relative to the magnet calibration measurement)
typically should not exceed the 1% to 2% level. A 1% deviation would be
considered high by magnet engineers. However, in practice it is common to
see such a level of
∆K
K in optics fitting results and usually it is applicable for
optics correction.
Imposing constraints with Levenberg-Marquardt method:
Adding a positive definite diagonal matrix to the covariance matrix in
Eq. (4.40) to improve the convergence is a common practice in nonlinear
least-square fitting problems. This method was first proposed by Levenberg
and later rediscovered by Marquardt and is referred to as the LevenbergMarquardt (L-M) method [79, 85]. In nonlinear least-square problems, when
the present solution is far away from the minimum, the Gauss-Newton method
could fail to converge because the linear expansion of the residual vector may
not be sufficiently accurate. In such a case, a sensible approach is to use the
gradient-descent method to advance the solution toward the local direction of
χ
2 reduction.
The original Levenberg method proposes to modify the Gauss-Newton
solution to
∆p = −(J
T J + λI)
−1 J
T r 0 ,
(4.41)
where I is the identity matrix and λ is a scalar coefficient, which is to be
adjusted in each iteration. In an iteration, if the increment ∆p does not lead
to a reduction of χ
2 , the λ value is increased (e.g., by ×10); if it brings down χ
2
by a reasonable amount, λ is reduced. With a large λ, the algorithm behaves
like the gradient-descent method since the gradient of χ
2 with respect to ∆p
is 2J
T r 0 (see Eq. (4.29)). When λ shrinks to near zero, the algorithm becomes
the Gauss-Newton method.
A modified version of the L-M method is in the form
∆p = −(J
T J + λdiag(J
T J))
−1 J
T r 0 ,
(4.42)
where diag(J
T J) is a diagonal matrix that keeps only the diagonal elements
of the matrix J
T J while setting all other elements to zero. The modified L-M
method scales the increment in each axis, such that a larger step can be made
in a direction with a smaller gradient.
Comparing the solutions of the L-M method to Eq. (4.40), it is clear that
the L-M method is equivalent to imposing constraints to the individual fitting
parameters. The corresponding cost functions for the two forms of L-M method
are
Original: λ
Nq
i=1 ∆p
2
i ,
Modified: λ
Nq
i=1 J
T
i J i ∆p
2
i ,
in steps while observing the ∆K values in the fitting solution and the resulting χ
2 . A set of weight factors can be accepted if the fitting solution brings
χ
2 down to the same level as the case without constraints (or the case with
very weak constraints if no solution can be found without constraints) with
reasonable values of
∆K
K . Unless there is a justifiable reason, the fitted
∆K
K
for a quadrupole parameter (relative to the magnet calibration measurement)
typically should not exceed the 1% to 2% level. A 1% deviation would be
considered high by magnet engineers. However, in practice it is common to
see such a level of
∆K
K in optics fitting results and usually it is applicable for
optics correction.
Imposing constraints with Levenberg-Marquardt method:
Adding a positive definite diagonal matrix to the covariance matrix in
Eq. (4.40) to improve the convergence is a common practice in nonlinear
least-square fitting problems. This method was first proposed by Levenberg
and later rediscovered by Marquardt and is referred to as the LevenbergMarquardt (L-M) method [79, 85]. In nonlinear least-square problems, when
the present solution is far away from the minimum, the Gauss-Newton method
could fail to converge because the linear expansion of the residual vector may
not be sufficiently accurate. In such a case, a sensible approach is to use the
gradient-descent method to advance the solution toward the local direction of
χ
2 reduction.
The original Levenberg method proposes to modify the Gauss-Newton
solution to
∆p = −(J
T J + λI)
−1 J
T r 0 ,
(4.41)
where I is the identity matrix and λ is a scalar coefficient, which is to be
adjusted in each iteration. In an iteration, if the increment ∆p does not lead
to a reduction of χ
2 , the λ value is increased (e.g., by ×10); if it brings down χ
2
by a reasonable amount, λ is reduced. With a large λ, the algorithm behaves
like the gradient-descent method since the gradient of χ
2 with respect to ∆p
is 2J
T r 0 (see Eq. (4.29)). When λ shrinks to near zero, the algorithm becomes
the Gauss-Newton method.
A modified version of the L-M method is in the form
∆p = −(J
T J + λdiag(J
T J))
−1 J
T r 0 ,
(4.42)
where diag(J
T J) is a diagonal matrix that keeps only the diagonal elements
of the matrix J
T J while setting all other elements to zero. The modified L-M
method scales the increment in each axis, such that a larger step can be made
in a direction with a smaller gradient.
Comparing the solutions of the L-M method to Eq. (4.40), it is clear that
the L-M method is equivalent to imposing constraints to the individual fitting
parameters. The corresponding cost functions for the two forms of L-M method
are
Original: λ
Nq
i=1 ∆p
2
i ,
Modified: λ
Nq
i=1 J
T
i J i ∆p
2
i ,