184
11 Optimal Control Theory
Since the variations δx, δu, and δp are independent, the terms in the square brackets must vanish individually. Collecting these equations, we find the equivalent of
Hamilton’s equations, which are the “equations of motion”
˙
p = −
∂g
∂x
− p
t ∂a
∂x
˙
x = a(x, u)
(11.39)
0 =
∂g
∂u
+ p
t ∂a
∂u
for a Hamiltonian H(x, u, p), given by
H(x, u, p) = g(x, u) + p
t a .
(11.40)
With this definition of the Hamiltonian, we can finally write the equations of motion
from (11.39) as
˙
p = −
∂H
∂x
˙
x =
∂H
∂p
0 =
∂H
∂u
.
(11.41)
Now we see that the costates p, originally introduced as Lagrange multipliers,
assume the role of the momentum corresponding to the states x. The Hamiltonian
H, defined in (11.40), reminds of a Legendre transform of g. Moreover, minimizing
the objective functional is now written as the requirement of the partial derivative of
H(x, u, p) with respect to u (the gradient) being zero.
So, what have we gained by rephrasing the problem in a Hamiltonian framework?
First, we found a consistent way to handle the optimization when the state vector x
moves around while we change the controls u. Second, the minimization now looks
like a normal minimization; we only have to calculate the gradient of the Hamiltonian
with respect to the controls u and set the result equal to zero.
In order to illustrate this new formalism, we will have a second look at the donkey
from Sect. 11.2.
11.5 Donkey Revisited
To make the problem as simple as possible (but not simpler than that), we require
the donkey to arrive at x = l at time t = t f and minimize the integral of the power
expended, which is proportional to g(x, u) = u
2
/2 integrated from t = 0 to t = t f .
Moreover, we assume that the variables are suitably scaled, such that m = 1 and
F = u. The equations of motion from (11.20) then assume the form
˙
x 1 = x 2
˙
x 2 = −αx 2 + u
(11.42)
11 Optimal Control Theory
Since the variations δx, δu, and δp are independent, the terms in the square brackets must vanish individually. Collecting these equations, we find the equivalent of
Hamilton’s equations, which are the “equations of motion”
˙
p = −
∂g
∂x
− p
t ∂a
∂x
˙
x = a(x, u)
(11.39)
0 =
∂g
∂u
+ p
t ∂a
∂u
for a Hamiltonian H(x, u, p), given by
H(x, u, p) = g(x, u) + p
t a .
(11.40)
With this definition of the Hamiltonian, we can finally write the equations of motion
from (11.39) as
˙
p = −
∂H
∂x
˙
x =
∂H
∂p
0 =
∂H
∂u
.
(11.41)
Now we see that the costates p, originally introduced as Lagrange multipliers,
assume the role of the momentum corresponding to the states x. The Hamiltonian
H, defined in (11.40), reminds of a Legendre transform of g. Moreover, minimizing
the objective functional is now written as the requirement of the partial derivative of
H(x, u, p) with respect to u (the gradient) being zero.
So, what have we gained by rephrasing the problem in a Hamiltonian framework?
First, we found a consistent way to handle the optimization when the state vector x
moves around while we change the controls u. Second, the minimization now looks
like a normal minimization; we only have to calculate the gradient of the Hamiltonian
with respect to the controls u and set the result equal to zero.
In order to illustrate this new formalism, we will have a second look at the donkey
from Sect. 11.2.
11.5 Donkey Revisited
To make the problem as simple as possible (but not simpler than that), we require
the donkey to arrive at x = l at time t = t f and minimize the integral of the power
expended, which is proportional to g(x, u) = u
2
/2 integrated from t = 0 to t = t f .
Moreover, we assume that the variables are suitably scaled, such that m = 1 and
F = u. The equations of motion from (11.20) then assume the form
˙
x 1 = x 2
˙
x 2 = −αx 2 + u
(11.42)