Chapter 11
Optimal Control Theory
Abstract After introducing the Solow and Robinson-Crusoe models as examples
of real business cycle models—analogous to the equations of motion in physics—we
derive the Bellman equation to find a control law for a parameter that optimizes a
performance measure. A simple mechanical model, based on a donkey pulling a mass
across rough ground, illustrates the basics of the state-space formalism in optimal
control theory. Following a review of the relation between Lagrange and Hamilton
functions in classical mechanics, this chapter derives Hamilton’s equations for general dynamical systems that minimize a performance measure. Here similarities to
Hamilton’s principle of minimizing the action are pertinent. At this point we can
use the newly-developed methods to optimize the donkey’s progress, before using
the same methods to derive the Riccati equation and analyze linear quadratic regulators that are subsequently used to control a Robinson-Crusoe economy close to its
equilibrium.
In this chapter we will discuss methods to control parameters that affect a dynamical
system in order to achieve some desirable objective. Examples from engineering are
control valves that regulate the flow rate of liquids needed to maintain a chemical
reaction; or centrifugal governors, regulators that automatically control the speed of
steam engines, the ancestor of the cruise controller found in modern cars. Another
group of systems that can be controlled are macroeconomic systems. Central banks
control the interest rate at which commercial banks can borrow money. This directly
affects the availability of loans to companies, which they use to expand, for example,
by opening an additional factory. As a consequence, the unemployment rate drops,
because the new jobs in the factories must be filled. The overall effect of the actions
of the central bank is (hopefully) an increase of the gross domestic product and the
wealth of the population as a whole. At least that is the theory, and a prerequisite
to analyze this theory is a mathematical model for the dynamics of the system.
In physics, the models are described by equations of motion. In economics, rate
equations that describe the dynamics are at the heart of so-called real business cycle
models [1].
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021
V. Ziemann, Physics and Finance, Undergraduate Lecture Notes in Physics,
https://doi.org/10.1007/978-3-030-63643-2_11
171
Optimal Control Theory
Abstract After introducing the Solow and Robinson-Crusoe models as examples
of real business cycle models—analogous to the equations of motion in physics—we
derive the Bellman equation to find a control law for a parameter that optimizes a
performance measure. A simple mechanical model, based on a donkey pulling a mass
across rough ground, illustrates the basics of the state-space formalism in optimal
control theory. Following a review of the relation between Lagrange and Hamilton
functions in classical mechanics, this chapter derives Hamilton’s equations for general dynamical systems that minimize a performance measure. Here similarities to
Hamilton’s principle of minimizing the action are pertinent. At this point we can
use the newly-developed methods to optimize the donkey’s progress, before using
the same methods to derive the Riccati equation and analyze linear quadratic regulators that are subsequently used to control a Robinson-Crusoe economy close to its
equilibrium.
In this chapter we will discuss methods to control parameters that affect a dynamical
system in order to achieve some desirable objective. Examples from engineering are
control valves that regulate the flow rate of liquids needed to maintain a chemical
reaction; or centrifugal governors, regulators that automatically control the speed of
steam engines, the ancestor of the cruise controller found in modern cars. Another
group of systems that can be controlled are macroeconomic systems. Central banks
control the interest rate at which commercial banks can borrow money. This directly
affects the availability of loans to companies, which they use to expand, for example,
by opening an additional factory. As a consequence, the unemployment rate drops,
because the new jobs in the factories must be filled. The overall effect of the actions
of the central bank is (hopefully) an increase of the gross domestic product and the
wealth of the population as a whole. At least that is the theory, and a prerequisite
to analyze this theory is a mathematical model for the dynamics of the system.
In physics, the models are described by equations of motion. In economics, rate
equations that describe the dynamics are at the heart of so-called real business cycle
models [1].
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021
V. Ziemann, Physics and Finance, Undergraduate Lecture Notes in Physics,
https://doi.org/10.1007/978-3-030-63643-2_11
171