Chapter 5
An Introduction to Imprecise Markov
Chains
Thomas Krak
Abstract Stochastic processes in general provide a popular framework for modelling uncertainty about the evolution of dynamical systems. The theory of Markov
chains uses a number of crucial assumptions about the (in)dependence of such
a process on its history that make their analysis tractable. In practice however,
the parameters of a Markov chain may not be known exactly, or there may exist
doubt as to the applicability of these assumptions to the system under study.
This chapter presents an introduction to imprecise Markov chains, which are a
robust generalisation of these models that may be used when parameters are not
known exactly or when such assumptions could be violated. Their treatment is
grounded in the theory of imprecise probabilities. The generalised model can be
interpreted as a set of (traditional) stochastic processes, which may or may not be
Markovian and which may have different and varying parameter values. Inferences
are then performed to ensure robustness with respect to variations within this set.
This chapter assumes no advanced familiarity with Markov chains or imprecise
probability theory. It aims to develop an intuitive and graphical understanding of
(imprecise) Markov chains in discrete and in continuous time.
Keywords Imprecise probabilities · Model uncertainty · Stochastic processes ·
Imprecise Markov chains
5.1 Introduction
In many areas of science and engineering, we are interested in modelling uncertainty
about the behaviour of dynamical systems, that is, systems whose state changes
as time passes. For instance, we may want to model the evolution of the spatial
trajectories of a system in motion; or the performance and reliability of a complex
T. Krak ()
IDLab, Ghent University, Ghent, Belgium
e-mail: thomas.krak@ugent.be
© Springer Nature Switzerland AG 2021
M. Vasile (ed.), Optimization Under Uncertainty with Applications to Aerospace
Engineering, https://doi.org/10.1007/978-3-030-60166-9_5
141
An Introduction to Imprecise Markov
Chains
Thomas Krak
Abstract Stochastic processes in general provide a popular framework for modelling uncertainty about the evolution of dynamical systems. The theory of Markov
chains uses a number of crucial assumptions about the (in)dependence of such
a process on its history that make their analysis tractable. In practice however,
the parameters of a Markov chain may not be known exactly, or there may exist
doubt as to the applicability of these assumptions to the system under study.
This chapter presents an introduction to imprecise Markov chains, which are a
robust generalisation of these models that may be used when parameters are not
known exactly or when such assumptions could be violated. Their treatment is
grounded in the theory of imprecise probabilities. The generalised model can be
interpreted as a set of (traditional) stochastic processes, which may or may not be
Markovian and which may have different and varying parameter values. Inferences
are then performed to ensure robustness with respect to variations within this set.
This chapter assumes no advanced familiarity with Markov chains or imprecise
probability theory. It aims to develop an intuitive and graphical understanding of
(imprecise) Markov chains in discrete and in continuous time.
Keywords Imprecise probabilities · Model uncertainty · Stochastic processes ·
Imprecise Markov chains
5.1 Introduction
In many areas of science and engineering, we are interested in modelling uncertainty
about the behaviour of dynamical systems, that is, systems whose state changes
as time passes. For instance, we may want to model the evolution of the spatial
trajectories of a system in motion; or the performance and reliability of a complex
T. Krak ()
IDLab, Ghent University, Ghent, Belgium
e-mail: thomas.krak@ugent.be
© Springer Nature Switzerland AG 2021
M. Vasile (ed.), Optimization Under Uncertainty with Applications to Aerospace
Engineering, https://doi.org/10.1007/978-3-030-60166-9_5
141