5 An Introduction to Imprecise Markov Chains
159
The reason that this property is so important is that it provides a sufficient
condition for the long-term behaviour of a Markov chain to converge to a stationary
distribution, regardless of the state in which it started:
Theorem 5.2 Let {X t } t∈N 0 be a discrete-time homogeneous Markov chain, and let
T be its associated transition matrix. Let this Markov chain be regular. Then there
is a probability mass function P ∞ : X → R ≥0 such that, for all x, y ∈ X ,
P ∞ (y) = lim
n→+∞
T
n (x, y) .
5.3 Imprecise Discrete-Time Markov Chains
We will now move on to the discussion surrounding imprecise (discrete-time)
Markov chains (IDTMCs). So, we still consider the time-dimension T = N 0 . We
will generalise each of the representations that we previously encountered to this
new setting, where we roughly follow the same order as in Sect. 5.2.
So, let us start with the ‘measure-theoretic’ representation of imprecise stochastic
processes. In this setting, we consider a set P of probability measures on the
measurable space of paths (Ω, F ). Then for each P ∈ P, we have a probability
space (Ω, F , P ), to which we can associate the precise stochastic process {X t } t∈N 0
as in Definition 5.1. For any function f ∈ L (X n ), n ∈ N, we can express the
expected value on the n time points t ⊂ N 0 as E P [f (X t )] as in Sect. 5.2. Recall
from Chap. 2 that in this imprecise probabilistic context, we are more generally
interested in the lower and upper expectation of f , which are defined, respectively,
as
E P
f (X t )
:= inf
P ∈P
E P
f (X t )
and E P
f (X t )
:= sup
P ∈P
E P
f (X t )
.
We briefly recall the well-known conjugacy relation E P
f (X t )
= −E P
−f (X t )
,
from which it follows that we can present the remainder of this discussion entirely in
terms of lower expectations; any corresponding results on upper expectations follow
directly through this relation.
Slightly more generally than the above, we will focus on conditional lower
expectations. Similar to the precise case that we discussed before, these are defined
for any f ∈ L (X n+m ), n, m ∈ N, any s, t ⊂ N 0 such that s and t are of length n
and m, respectively, and any x s ∈ X n , as
E P
f (X s , X t )
X s = x s
:= inf
P ∈P
E P
f (X s , X t )
X s = x s
,
whenever E P [I x s (X s )] > 0. In this last condition, I x s is the indicator of x s ; for all
y s ∈ X n , I x s (y s ) := 1 if x s = y s and I x s (y s ) := 0, otherwise. Note that then
159
The reason that this property is so important is that it provides a sufficient
condition for the long-term behaviour of a Markov chain to converge to a stationary
distribution, regardless of the state in which it started:
Theorem 5.2 Let {X t } t∈N 0 be a discrete-time homogeneous Markov chain, and let
T be its associated transition matrix. Let this Markov chain be regular. Then there
is a probability mass function P ∞ : X → R ≥0 such that, for all x, y ∈ X ,
P ∞ (y) = lim
n→+∞
T
n (x, y) .
5.3 Imprecise Discrete-Time Markov Chains
We will now move on to the discussion surrounding imprecise (discrete-time)
Markov chains (IDTMCs). So, we still consider the time-dimension T = N 0 . We
will generalise each of the representations that we previously encountered to this
new setting, where we roughly follow the same order as in Sect. 5.2.
So, let us start with the ‘measure-theoretic’ representation of imprecise stochastic
processes. In this setting, we consider a set P of probability measures on the
measurable space of paths (Ω, F ). Then for each P ∈ P, we have a probability
space (Ω, F , P ), to which we can associate the precise stochastic process {X t } t∈N 0
as in Definition 5.1. For any function f ∈ L (X n ), n ∈ N, we can express the
expected value on the n time points t ⊂ N 0 as E P [f (X t )] as in Sect. 5.2. Recall
from Chap. 2 that in this imprecise probabilistic context, we are more generally
interested in the lower and upper expectation of f , which are defined, respectively,
as
E P
f (X t )
:= inf
P ∈P
E P
f (X t )
and E P
f (X t )
:= sup
P ∈P
E P
f (X t )
.
We briefly recall the well-known conjugacy relation E P
f (X t )
= −E P
−f (X t )
,
from which it follows that we can present the remainder of this discussion entirely in
terms of lower expectations; any corresponding results on upper expectations follow
directly through this relation.
Slightly more generally than the above, we will focus on conditional lower
expectations. Similar to the precise case that we discussed before, these are defined
for any f ∈ L (X n+m ), n, m ∈ N, any s, t ⊂ N 0 such that s and t are of length n
and m, respectively, and any x s ∈ X n , as
E P
f (X s , X t )
X s = x s
:= inf
P ∈P
E P
f (X s , X t )
X s = x s
,
whenever E P [I x s (X s )] > 0. In this last condition, I x s is the indicator of x s ; for all
y s ∈ X n , I x s (y s ) := 1 if x s = y s and I x s (y s ) := 0, otherwise. Note that then