5 An Introduction to Imprecise Markov Chains
143
when the time-dimension is discrete. Specifically, we cover the representation using
probability trees, in Sect. 5.2.1; using Bayesian networks, in Sect. 5.2.2 and using
transition graphs, in Sect. 5.2.3.
Once we have developed these different ways of reasoning about discrete-time
processes, we generalise the discussion to imprecise discrete-time processes in
Sect. 5.3. We use the previously developed graphical notions to provide intuition
about how to reason and compute inferences using these models. The treatment of
(imprecise) continuous-time processes is largely postponed until Sect. 5.4. Here the
graphical and intuitive representations largely break down, but we can then use the
previously developed understanding of the discrete-time case to reason about these
models. To keep the main text as readable as possible, the discussion of the literature
on which the material in this chapter is based is deferred to Sect. 5.5.
5.2 (Precise) Stochastic Processes
We will start the exposition around stochastic processes in a relatively general and
abstract sense but will quickly make things more specific. Throughout the remainder
of this chapter, we will consider some fixed abstract state-space X . A state is
an element x ∈ X and represents uniquely the relevant information about the
underlying system that we are interested in modelling. So as not to complicate
matters, we will assume throughout that X is finite, so that we can identify it
without loss of generality as the set X = {1, . . . , k} ⊂ N. Note that here and
in what follows, we denote with N the natural numbers and will write N 0 := N ∪ {0}
when we include zero. Furthermore, the real numbers are written R, the nonnegative reals are R ≥0 and the positive reals are R >0 .
Because we are interested in modelling a system whose state x ∈ X changes
over time, we next identify some time-dimension T. A crucial choice to be made
later on is whether we are considering processes in discrete-time, in which case we
identify T = N 0 , or processes in continuous-time, in which case T = R ≥0 . For now
we simply keep the discussion general without making this identification.
With the state-space and time-dimension in place, it now makes sense to talk
about the realisation of some (yet to be identified) stochastic process. Such a
realisation is also called a sample path, and it is a function ω : T → X . So,
this ω describes for each point in time t ∈ T the state ω(t) ∈ X that the system
was in at that time. We collect in the set Ω all these sample paths. For technical
reasons, it is sometimes required to restrict attention to paths that satisfy sufficient
smoothness conditions; for instance, when T = R ≥0 , it is common practice to let
Ω only contain càdlàg functions, that is, paths ω(t) that are right-continuous and
whose left-sided limits exist everywhere.
This set Ω thus contains all possible ways in which the system might behave
over time; it can therefore be considered an outcome space of a stochastic model.
Formally, we will consider some abstract underlying probability space (Ω, F , P ),
where F is some appropriate σ -algebra on Ω and where P is a probability measure
143
when the time-dimension is discrete. Specifically, we cover the representation using
probability trees, in Sect. 5.2.1; using Bayesian networks, in Sect. 5.2.2 and using
transition graphs, in Sect. 5.2.3.
Once we have developed these different ways of reasoning about discrete-time
processes, we generalise the discussion to imprecise discrete-time processes in
Sect. 5.3. We use the previously developed graphical notions to provide intuition
about how to reason and compute inferences using these models. The treatment of
(imprecise) continuous-time processes is largely postponed until Sect. 5.4. Here the
graphical and intuitive representations largely break down, but we can then use the
previously developed understanding of the discrete-time case to reason about these
models. To keep the main text as readable as possible, the discussion of the literature
on which the material in this chapter is based is deferred to Sect. 5.5.
5.2 (Precise) Stochastic Processes
We will start the exposition around stochastic processes in a relatively general and
abstract sense but will quickly make things more specific. Throughout the remainder
of this chapter, we will consider some fixed abstract state-space X . A state is
an element x ∈ X and represents uniquely the relevant information about the
underlying system that we are interested in modelling. So as not to complicate
matters, we will assume throughout that X is finite, so that we can identify it
without loss of generality as the set X = {1, . . . , k} ⊂ N. Note that here and
in what follows, we denote with N the natural numbers and will write N 0 := N ∪ {0}
when we include zero. Furthermore, the real numbers are written R, the nonnegative reals are R ≥0 and the positive reals are R >0 .
Because we are interested in modelling a system whose state x ∈ X changes
over time, we next identify some time-dimension T. A crucial choice to be made
later on is whether we are considering processes in discrete-time, in which case we
identify T = N 0 , or processes in continuous-time, in which case T = R ≥0 . For now
we simply keep the discussion general without making this identification.
With the state-space and time-dimension in place, it now makes sense to talk
about the realisation of some (yet to be identified) stochastic process. Such a
realisation is also called a sample path, and it is a function ω : T → X . So,
this ω describes for each point in time t ∈ T the state ω(t) ∈ X that the system
was in at that time. We collect in the set Ω all these sample paths. For technical
reasons, it is sometimes required to restrict attention to paths that satisfy sufficient
smoothness conditions; for instance, when T = R ≥0 , it is common practice to let
Ω only contain càdlàg functions, that is, paths ω(t) that are right-continuous and
whose left-sided limits exist everywhere.
This set Ω thus contains all possible ways in which the system might behave
over time; it can therefore be considered an outcome space of a stochastic model.
Formally, we will consider some abstract underlying probability space (Ω, F , P ),
where F is some appropriate σ -algebra on Ω and where P is a probability measure