Appendix I
Optimization Techniques
There is “. . . a general tendency in driven many-particle
systems toward self-organization into states formed through
exceptionally reliable absorption and dissipation of work energy
from the surrounding environment.”
— Jeremy England, Nikolai Peruvov, and Robert Marshland
I.1 The Lagrange Multiplier Method
There are many examples of optimization and minimization related to life systems
such as those in mathematical biology, theoretical biochemistry, network analysis,
neuronal systems, statistical mechanics, and complex systems.
Joseph Louis Lagrange formulated a beautiful method for such minimization
when there are constraints among the variables. The method is particularly useful if
the constraints are awkward or difficult to solve for one subset of variables in terms
of the others.
Lagrange showed that the minimization of a function f (x 1 , x 2 , · · · , x N ) under
a number n < N of constraints among the values in {x} given by a set of relations
{κ l (x) = 0 1 ≤ l ≤ n} is equivalent to the unconstrained minimization of
F (x, λ) ≡ f (x) +
l λ l κ l (x), i.e. a minimum of f with constraints occurs at those
values of {x} which make the slopes of F vanish:
∂
∂x k
F (x) =
∂
∂x k
f (x) +
n
l=1
λ l
∂
∂x k
κ l (x) = 0 .
(I.1)
These N equations, plus the n constraint relations κ l (x) = 0 can be solved for the
N + n values in the set {x, λ}. Now, n of these solutions can be used to eliminate
the so-called Lagrange multipliers {λ}, giving the N solutions in the set {x} that
may minimize, maximize, or reach a boundary in the range of {x} for the cost
function f under constraints. The locations where the derivatives of F vanish within
the boundaries are called ’critical’ points, and may be local minima, maxima, or
‘saddle’ points of F . A condition for the minima can be generated as follows:
At each critical point, solve the ‘secular’ equation det
F ij − a i δ ij
= 0 for the
© Springer Nature Switzerland AG 2020
W. C. Parke, Biophysics, https://doi.org/10.1007/978-3-030-44146-3
675
Optimization Techniques
There is “. . . a general tendency in driven many-particle
systems toward self-organization into states formed through
exceptionally reliable absorption and dissipation of work energy
from the surrounding environment.”
— Jeremy England, Nikolai Peruvov, and Robert Marshland
I.1 The Lagrange Multiplier Method
There are many examples of optimization and minimization related to life systems
such as those in mathematical biology, theoretical biochemistry, network analysis,
neuronal systems, statistical mechanics, and complex systems.
Joseph Louis Lagrange formulated a beautiful method for such minimization
when there are constraints among the variables. The method is particularly useful if
the constraints are awkward or difficult to solve for one subset of variables in terms
of the others.
Lagrange showed that the minimization of a function f (x 1 , x 2 , · · · , x N ) under
a number n < N of constraints among the values in {x} given by a set of relations
{κ l (x) = 0 1 ≤ l ≤ n} is equivalent to the unconstrained minimization of
F (x, λ) ≡ f (x) +
l λ l κ l (x), i.e. a minimum of f with constraints occurs at those
values of {x} which make the slopes of F vanish:
∂
∂x k
F (x) =
∂
∂x k
f (x) +
n
l=1
λ l
∂
∂x k
κ l (x) = 0 .
(I.1)
These N equations, plus the n constraint relations κ l (x) = 0 can be solved for the
N + n values in the set {x, λ}. Now, n of these solutions can be used to eliminate
the so-called Lagrange multipliers {λ}, giving the N solutions in the set {x} that
may minimize, maximize, or reach a boundary in the range of {x} for the cost
function f under constraints. The locations where the derivatives of F vanish within
the boundaries are called ’critical’ points, and may be local minima, maxima, or
‘saddle’ points of F . A condition for the minima can be generated as follows:
At each critical point, solve the ‘secular’ equation det
F ij − a i δ ij
= 0 for the
© Springer Nature Switzerland AG 2020
W. C. Parke, Biophysics, https://doi.org/10.1007/978-3-030-44146-3
675