676
I Optimization Techniques
‘eigenvalues’ a i , where F ij ≡ ∂ 2 F /(∂x i ∂x j ). If all the a i are positive, the critical
point is a minimum of F .
To get a flavor of the proof, consider the case of just two variables, x and y. We
wish to minimize the function f (x, y), with a constraint κ(x, y) = 0. Suppose we
solve the constraint equation to get y = h(x) and substitute in f to get f (x, h(x)).
We then find where this new function of x reaches a minimum, i.e. where df/dx =
0. We can express this derivative by
∂f (x, y)
∂x
+
∂f (x, y)
∂y
∂h(x)
∂x
y=h(x)
= 0 .
(I.2)
We can find ∂h(x)/∂x by differentiating the identity κ(x, h(x)) ≡ 0 to get
∂κ(x, y)
∂x
+
∂κ(x, y)
∂y
∂h(x)
∂x
y=h(x)
≡ 0 .
(I.3)
so
∂h(x)
∂x
= −
∂κ/∂x
∂κ/∂y
.
(I.4)
Substituting in Eq. (I.2) gives an expression for minimization in the form
∂f (x, y)
∂x
−
∂f (x, y)
∂y
∂κ/∂x
∂κ/∂y
= 0 .
(I.5)
Now consider the slopes of the function F (x, y, λ) = f (x, y)+λκ(x, y) without
any constraint on x, y except for possible boundary conditions. The three slopes
will be
∂f (x, y)
∂x
+ λ
∂κ(x, y)
∂x
= 0 ,
(I.6)
∂f (x, y)
∂y
+ λ
∂κ(x, y)
∂y
= 0 ,
(I.7)
κ(x, y) = 0.
(I.8)
If we put λ from the second relation into the first, we get the condition Eq. (I.5).
So the unconstrained minimization of F (x, y, λ) gives the same solution as the
constrained minimization of f (x, y).