2.3 Mathematical Tools
57
non-smooth functions related by a Legendre transformation
y(x) = d x f (x) and x(y) = d y f
∗
(y) and y x = f (x) + f
∗
(y). (2.125)
Here, for non-smooth functions, d x f (x) denotes the set of sub-derivatives, i.e. the
sub-differential
11 of f (x) with respect to x, and d y f
∗
(y) denotes the set of subderivatives, i.e. the sub-differential of f
∗
(y) with respect to y. As consequences of
the above properties (i) d x f (x) and d y f
∗
(y) are inverses of one another, and (ii)
applying Legendre transformation twice renders f (x) = f
∗∗
(x).
For a motivation consider the smooth y versus x diagram in Fig. 2.16 with the
dependence between y and x either given by y = y(x) or by x = x(y). Then f (x)
follows as the integral f (x) :=
x
0 y(x
) dx
, i.e. the area under the y = y(x) curve,
thus rendering ∂ x f = y(x), whereas the dual f
∗
(y) follows as the integral f
∗
(y) =
y
0 x(y
) dy
, i.e. the area under the x = x(y) curve, thus rendering ∂ y f
∗
= x(y).
Obviously f (x) and f
∗
(y) sum up to y x.
2.3.5 Constrained Optimization
General Setting
Generically, a constrained optimization problem consists in seeking the variable (or
function) x 0 that minimizes a function (or a functional) F(x), i.e.
x 0 = arg{min
x
F(x)}
(2.126)
typically under some equality and inequality constraints
h(x) = 0 and g(x) ≤ 0
(2.127)
(box constraints and other types of constraints shall not be considered here). Since in
the sequel this treatise is essentially concerned with problems restricted by inequality
constraints, the equality constraint h(x) = 0 shall not be considered further. Finally,
based on the McCauley brackets
2 g(x) := g(x) + |g(x)| ≥ 0
(2.128)
that extract the positive part of a function, the remaining inequality constraint g(x) ≤
0 may alternatively be expressed as (non-smooth) equality constraint
11 The sub-differential of a non-smooth convex function f = f (x) with x ∈ X is defined as the set
of sub-differentials d x f | x0 := {z| f (x) − f (x 0 ) − z [x − x 0 ] ≥ 0 ∀x ∈ X}. For smooth functions
the sub-differential degenerates to the ordinary derivative z = f .
57
non-smooth functions related by a Legendre transformation
y(x) = d x f (x) and x(y) = d y f
∗
(y) and y x = f (x) + f
∗
(y). (2.125)
Here, for non-smooth functions, d x f (x) denotes the set of sub-derivatives, i.e. the
sub-differential
11 of f (x) with respect to x, and d y f
∗
(y) denotes the set of subderivatives, i.e. the sub-differential of f
∗
(y) with respect to y. As consequences of
the above properties (i) d x f (x) and d y f
∗
(y) are inverses of one another, and (ii)
applying Legendre transformation twice renders f (x) = f
∗∗
(x).
For a motivation consider the smooth y versus x diagram in Fig. 2.16 with the
dependence between y and x either given by y = y(x) or by x = x(y). Then f (x)
follows as the integral f (x) :=
x
0 y(x
) dx
, i.e. the area under the y = y(x) curve,
thus rendering ∂ x f = y(x), whereas the dual f
∗
(y) follows as the integral f
∗
(y) =
y
0 x(y
) dy
, i.e. the area under the x = x(y) curve, thus rendering ∂ y f
∗
= x(y).
Obviously f (x) and f
∗
(y) sum up to y x.
2.3.5 Constrained Optimization
General Setting
Generically, a constrained optimization problem consists in seeking the variable (or
function) x 0 that minimizes a function (or a functional) F(x), i.e.
x 0 = arg{min
x
F(x)}
(2.126)
typically under some equality and inequality constraints
h(x) = 0 and g(x) ≤ 0
(2.127)
(box constraints and other types of constraints shall not be considered here). Since in
the sequel this treatise is essentially concerned with problems restricted by inequality
constraints, the equality constraint h(x) = 0 shall not be considered further. Finally,
based on the McCauley brackets
2 g(x) := g(x) + |g(x)| ≥ 0
(2.128)
that extract the positive part of a function, the remaining inequality constraint g(x) ≤
0 may alternatively be expressed as (non-smooth) equality constraint
11 The sub-differential of a non-smooth convex function f = f (x) with x ∈ X is defined as the set
of sub-differentials d x f | x0 := {z| f (x) − f (x 0 ) − z [x − x 0 ] ≥ 0 ∀x ∈ X}. For smooth functions
the sub-differential degenerates to the ordinary derivative z = f .