188
11 Optimal Control Theory
provided that the matrix R is invertible, which rules out zero eigenvalues of R and
the matrix must be positive definite. Replacing the controller u in the equations for
the states and costates, we arrive at
˙
x
˙
p
=
A −B R
−1 B
t
−Q
−A
t
x
p
.
(11.53)
Note that if all matrices A, B, R, and Q are constant, this is a set of linear ordinary
differential equations with constant coefficients, which is straightforward to solve.
Instead of attempting a direct numerical solution, we will seek to find a special
solution in which the state and costate vectors are related through a matrix K (t) with
p = K (t)x, because jointly with (11.52) this allows us to find a control law of the
form
u = −R
−1 B
t K x ,
(11.54)
which determines the required control value u directly derived from the current state
x. Thus it makes both x and u small, as required by the objective functional. Let us
now find K . From the definition
p = K (t)x
we obtain
˙
p = ˙
K x + K ˙
x .
(11.55)
Solving for ˙
K x, and repeatedly using (11.51) and (11.53), we arrive at
˙
K x = ˙
p − K ˙
x = ˙
p − K (Ax + Bu)
= −Qx − A
t p − K Ax + K B R
−1 B
t p
(11.56)
= −Qx − A
t K x − K Ax + K B R
−1 B
t K x .
Since this equation must be satisfied for all states x, we can omit the states x from
the equality and obtain the Riccati equation
˙
K = −Q − A
t K − K A + K B R
−1 B
t K ,
(11.57)
which is a condition for the matrix K (t) in order to satisfy (11.55). Solving this
non-linear matrix-valued equation is in general non-trivial, but if we succeed, K
determines the optimal-control law via (11.54). This is the controller that maintains
small values of x with the least control effort in the sense that the controllers u are
as small as possible, where the relative weight between the state x and controllers
u is encoded in the matrices R and Q. For time-invariant systems with all matrices
being constant, the steady-state solution with ˙
K = 0, can be found from solving
0 = −Q − A
t K − K A + K B R
−1 B
t K .
(11.58)
which is an algebraic equation. This makes it easier to solve for K than to determine
K from the full Riccati equation (11.57), which is a non-linear differential equation.
11 Optimal Control Theory
provided that the matrix R is invertible, which rules out zero eigenvalues of R and
the matrix must be positive definite. Replacing the controller u in the equations for
the states and costates, we arrive at
˙
x
˙
p
=
A −B R
−1 B
t
−Q
−A
t
x
p
.
(11.53)
Note that if all matrices A, B, R, and Q are constant, this is a set of linear ordinary
differential equations with constant coefficients, which is straightforward to solve.
Instead of attempting a direct numerical solution, we will seek to find a special
solution in which the state and costate vectors are related through a matrix K (t) with
p = K (t)x, because jointly with (11.52) this allows us to find a control law of the
form
u = −R
−1 B
t K x ,
(11.54)
which determines the required control value u directly derived from the current state
x. Thus it makes both x and u small, as required by the objective functional. Let us
now find K . From the definition
p = K (t)x
we obtain
˙
p = ˙
K x + K ˙
x .
(11.55)
Solving for ˙
K x, and repeatedly using (11.51) and (11.53), we arrive at
˙
K x = ˙
p − K ˙
x = ˙
p − K (Ax + Bu)
= −Qx − A
t p − K Ax + K B R
−1 B
t p
(11.56)
= −Qx − A
t K x − K Ax + K B R
−1 B
t K x .
Since this equation must be satisfied for all states x, we can omit the states x from
the equality and obtain the Riccati equation
˙
K = −Q − A
t K − K A + K B R
−1 B
t K ,
(11.57)
which is a condition for the matrix K (t) in order to satisfy (11.55). Solving this
non-linear matrix-valued equation is in general non-trivial, but if we succeed, K
determines the optimal-control law via (11.54). This is the controller that maintains
small values of x with the least control effort in the sense that the controllers u are
as small as possible, where the relative weight between the state x and controllers
u is encoded in the matrices R and Q. For time-invariant systems with all matrices
being constant, the steady-state solution with ˙
K = 0, can be found from solving
0 = −Q − A
t K − K A + K B R
−1 B
t K .
(11.58)
which is an algebraic equation. This makes it easier to solve for K than to determine
K from the full Riccati equation (11.57), which is a non-linear differential equation.