5 An Introduction to Imprecise Markov Chains
167
in contrast to matrices, we cannot index the ‘elements’ of the transition operator.
Specifically, using Theorem 5.3, we can interpret the condition as
0 <
T
n I y
(x) = E P
I y (X n )
X 0 = x
= sup
P ∈P
P (X n = y | X 0 = x) ,
for all x, y ∈ X and some n ∈ N. What regularity asks for, then, is for there to be
some n ∈ N such that is possible for all x, y ∈ X to move from x to y in exactly
n steps, according to some P ∈ P. In particular, the (precise) measure P for which
this needs to be possible can be different for every pair x, y ∈ X . Regularity for
IDTMCs then is in a sense a much weaker—easier to satisfy—condition than that for
precise Markov chains. Nevertheless, the condition is sufficient for the following:
Theorem 5.4 Let P be a homogeneous IDTMC that is separately specified and
regular, with associated lower transition operator T . Then, there is a unique lower
expectation operator E P [·(X +∞ )] : L (X ) → R such that, for all f ∈ L (X )
and all x ∈ X ,
E P
f (X +∞ )
= lim
n→+∞
E P
f (X n ) | X 0 = x
= lim
n→+∞
T
n f
(x) .
Furthermore, this is the unique T -invariant lower expectation on L (X ), meaning
that E P [f (X +∞ )] = E P
[T f ](X +∞ )
for all f ∈ L (X ).
5.4 Imprecise Continuous-Time Markov Chains
We now move on to the discussion about (imprecise) continuous-time Markov
chains. We have already encountered this setting several times in the preceding
discussions but have generally skipped over any details. Let us recall from Sect. 5.2
that continuous-time stochastic processes are identified with a time-dimension
T = R ≥0 and that the elements ω of the outcome space of paths Ω are maps
ω : R ≥0 → X . The measure-theoretic definition is then as before, where
we consider the abstract probability space (Ω, F , P ), and the stochastic process
{X t } t∈R ≥0 is a family of random variables on this space. Furthermore, measuretheoretic definitions of (homogeneous) continuous-time Markov chains (CTMCs)
have already been encountered in Definitions 5.5 and 5.6.
How, then, can these models be interpreted? Let us start by considering the
simplest case, viz., a precise and homogeneous Markov chain in continuous-time.
According to the previous definitions, this is a stochastic process such that
1. P (X t | X s 1 , . . . , X s n ) = P (X t | X s n ) for all s 1 < · · · < s n < t in R ≥0 , and
2. P (X t | X s ) = P (X t−s | X 0 ) for all s < t in R ≥0 .
The immediate difficulty of moving on from this abstract representation is that the
time-dimension is now, in a sense, too big to use any of the previous representations.
167
in contrast to matrices, we cannot index the ‘elements’ of the transition operator.
Specifically, using Theorem 5.3, we can interpret the condition as
0 <
T
n I y
(x) = E P
I y (X n )
X 0 = x
= sup
P ∈P
P (X n = y | X 0 = x) ,
for all x, y ∈ X and some n ∈ N. What regularity asks for, then, is for there to be
some n ∈ N such that is possible for all x, y ∈ X to move from x to y in exactly
n steps, according to some P ∈ P. In particular, the (precise) measure P for which
this needs to be possible can be different for every pair x, y ∈ X . Regularity for
IDTMCs then is in a sense a much weaker—easier to satisfy—condition than that for
precise Markov chains. Nevertheless, the condition is sufficient for the following:
Theorem 5.4 Let P be a homogeneous IDTMC that is separately specified and
regular, with associated lower transition operator T . Then, there is a unique lower
expectation operator E P [·(X +∞ )] : L (X ) → R such that, for all f ∈ L (X )
and all x ∈ X ,
E P
f (X +∞ )
= lim
n→+∞
E P
f (X n ) | X 0 = x
= lim
n→+∞
T
n f
(x) .
Furthermore, this is the unique T -invariant lower expectation on L (X ), meaning
that E P [f (X +∞ )] = E P
[T f ](X +∞ )
for all f ∈ L (X ).
5.4 Imprecise Continuous-Time Markov Chains
We now move on to the discussion about (imprecise) continuous-time Markov
chains. We have already encountered this setting several times in the preceding
discussions but have generally skipped over any details. Let us recall from Sect. 5.2
that continuous-time stochastic processes are identified with a time-dimension
T = R ≥0 and that the elements ω of the outcome space of paths Ω are maps
ω : R ≥0 → X . The measure-theoretic definition is then as before, where
we consider the abstract probability space (Ω, F , P ), and the stochastic process
{X t } t∈R ≥0 is a family of random variables on this space. Furthermore, measuretheoretic definitions of (homogeneous) continuous-time Markov chains (CTMCs)
have already been encountered in Definitions 5.5 and 5.6.
How, then, can these models be interpreted? Let us start by considering the
simplest case, viz., a precise and homogeneous Markov chain in continuous-time.
According to the previous definitions, this is a stochastic process such that
1. P (X t | X s 1 , . . . , X s n ) = P (X t | X s n ) for all s 1 < · · · < s n < t in R ≥0 , and
2. P (X t | X s ) = P (X t−s | X 0 ) for all s < t in R ≥0 .
The immediate difficulty of moving on from this abstract representation is that the
time-dimension is now, in a sense, too big to use any of the previous representations.