178
11 Optimal Control Theory
where the derivative of the capital ˙
k in the limit of t → 0 is given by ˙
k = (k t+1 −
k t )//t.
Let us now briefly introduce a generic notation that is widely used in the literature.
Discrete-time optimization problems are often defined using a framework, where the
dynamics of the system is described by difference equations x t+1 = F(x t ) + G(u t )
for the state variables x and the controllers u. Based on these equations of motion that
step the state variables forward in time, we then try to affect observables y t = H (x t ),
in such a way that an objective functional V [y, u] is minimized. Here V [y, u] depends
on the times series of both observables y and controllers u. For linear systems the
functions F, G, and H can be represented by matrices.
If the system is continuous in time, the difference equations are replaced by
differential equations of the form ˙
x = F(x) + G(u) for the state vectors x and the
controllers u. Likewise, observables y = H (x) that depend on the state vectors x
are introduced and the objective of the optimization is to minimize the objective
functional V [y, u], which is typically an integral over a function of y and u for
some time interval. Representing control problems by state variables, controllers
and observables is often referred to as state-space formalism. Such systems do not
only appear in economics, but also in physics and engineering. Let us therefore
consider a simple example, based on a donkey pulling a mass across rough ground.
11.2 Control and Feedback
The donkey pulling a mass m is shown in Fig. 11.3. The dynamics of the somewhat
simplified system is determined by the equations of motion for the mass
m ¨
x + α ˙
x = F ,
(11.18)
where α describes the friction between the mass and the rough ground. F is the
force—the controller—with which the donkey pulls the mass. To bring this equation
into the form mentioned at the end of the previous section, we convert it to state-space
form by introducing the state variables x 1 and x 2 , defined by
x = x 1
and
˙
x = ˙
x 1 = x 2 .
(11.19)
This allows us to write the equations of motion as a set of first-order differential
equations
˙
x 1 = x 2
˙
x 2 = −
α
m
x 2 +
F
m
,
(11.20)
which has the form given at the end of the previous section, provided we identify the
force as the controller u = F. Since this system of differential equations is linear, it
can also be written in matrix form
11 Optimal Control Theory
where the derivative of the capital ˙
k in the limit of t → 0 is given by ˙
k = (k t+1 −
k t )//t.
Let us now briefly introduce a generic notation that is widely used in the literature.
Discrete-time optimization problems are often defined using a framework, where the
dynamics of the system is described by difference equations x t+1 = F(x t ) + G(u t )
for the state variables x and the controllers u. Based on these equations of motion that
step the state variables forward in time, we then try to affect observables y t = H (x t ),
in such a way that an objective functional V [y, u] is minimized. Here V [y, u] depends
on the times series of both observables y and controllers u. For linear systems the
functions F, G, and H can be represented by matrices.
If the system is continuous in time, the difference equations are replaced by
differential equations of the form ˙
x = F(x) + G(u) for the state vectors x and the
controllers u. Likewise, observables y = H (x) that depend on the state vectors x
are introduced and the objective of the optimization is to minimize the objective
functional V [y, u], which is typically an integral over a function of y and u for
some time interval. Representing control problems by state variables, controllers
and observables is often referred to as state-space formalism. Such systems do not
only appear in economics, but also in physics and engineering. Let us therefore
consider a simple example, based on a donkey pulling a mass across rough ground.
11.2 Control and Feedback
The donkey pulling a mass m is shown in Fig. 11.3. The dynamics of the somewhat
simplified system is determined by the equations of motion for the mass
m ¨
x + α ˙
x = F ,
(11.18)
where α describes the friction between the mass and the rough ground. F is the
force—the controller—with which the donkey pulls the mass. To bring this equation
into the form mentioned at the end of the previous section, we convert it to state-space
form by introducing the state variables x 1 and x 2 , defined by
x = x 1
and
˙
x = ˙
x 1 = x 2 .
(11.19)
This allows us to write the equations of motion as a set of first-order differential
equations
˙
x 1 = x 2
˙
x 2 = −
α
m
x 2 +
F
m
,
(11.20)
which has the form given at the end of the previous section, provided we identify the
force as the controller u = F. Since this system of differential equations is linear, it
can also be written in matrix form