5 An Introduction to Imprecise Markov Chains
175
5.4.2 Limits of ICTMCs
Let us finally consider the long-term behaviour of a given homogeneous ICTMC P
with transition rate matrix set Q and associated lower transition rate operator Q; we
assume these to be fixed in the remainder of this section. What, then, can we say
about the lower expectation of a function as time goes to infinity?
Recall that, in the discrete-time case, Theorem 5.4 established a sufficient
condition for such a lower expectation to converge. This condition was regularity of
the IDTMC. Essentially, this meant that it was possible for the IDTMC to move from
any state to any other state, in exactly n steps, for some n ∈ N. In the continuoustime case that we consider here, there is a similar condition: upper reachability
between all pairs of states.
We first remark that this condition is defined using the conjugate upper transition
rate operator defined as Qf := −Q(−f ) for all f ∈ L (X ). The definition of
upper reachability is then analogous to that of accessibility in discrete-time but is
instead defined using the transition rates, rather than probabilities:
Definition 5.19 Let P be an ICTMC with associated upper transition rate operator
Q, as defined above. For any two states x, y ∈ X , y is said to be upper reachable
from x, if there is a sequence x 0 , . . . , x n ∈ X , n ∈ N, such that x 0 = x, x n = y
and, for all i ∈ {1, . . . , n}, it holds that x i = x i−1 and
Q I x i
(x i−1 ) > 0.
Let us in particular consider the final condition in this definition. From the conjugacy
between the lower and upper transition rate operators, and the definition of the
former, we can rewrite this requirement as saying that
0 <
Q I x i
(x i−1 ) = sup
Q∈Q
Q I x i
(x i−1 ) = sup
Q∈Q
Q(x i−1 , x i ) .
Thus, upper reachability of y, from x, requires that there exists a sequence of states
from x to y such that, at each step in this sequence, there is some transition rate
matrix Q ∈ Q which assigns strictly positive ‘speed’ of moving from the current
state in this sequence, to the next one. In other words, it should be possible for
these transitions to happen according to some of the models in our set P, but not
necessarily all, and there can be a different model allowing for this possibility at
each step. This can now be used to state the following result:
Theorem 5.7 Let P be an ICTMC and suppose that, for all x, y ∈ X , y is upper
reachable from x. Then, there is a unique lower expectation operator E P
·(X +∞ )
:
L (X ) → R such that, for all f ∈ L (X ) and all x ∈ X ,
E P
f (X +∞ )
= lim
t→+∞
E P
f (X t )
X 0 = x
= lim
t→+∞
T t f
(x) .
175
5.4.2 Limits of ICTMCs
Let us finally consider the long-term behaviour of a given homogeneous ICTMC P
with transition rate matrix set Q and associated lower transition rate operator Q; we
assume these to be fixed in the remainder of this section. What, then, can we say
about the lower expectation of a function as time goes to infinity?
Recall that, in the discrete-time case, Theorem 5.4 established a sufficient
condition for such a lower expectation to converge. This condition was regularity of
the IDTMC. Essentially, this meant that it was possible for the IDTMC to move from
any state to any other state, in exactly n steps, for some n ∈ N. In the continuoustime case that we consider here, there is a similar condition: upper reachability
between all pairs of states.
We first remark that this condition is defined using the conjugate upper transition
rate operator defined as Qf := −Q(−f ) for all f ∈ L (X ). The definition of
upper reachability is then analogous to that of accessibility in discrete-time but is
instead defined using the transition rates, rather than probabilities:
Definition 5.19 Let P be an ICTMC with associated upper transition rate operator
Q, as defined above. For any two states x, y ∈ X , y is said to be upper reachable
from x, if there is a sequence x 0 , . . . , x n ∈ X , n ∈ N, such that x 0 = x, x n = y
and, for all i ∈ {1, . . . , n}, it holds that x i = x i−1 and
Q I x i
(x i−1 ) > 0.
Let us in particular consider the final condition in this definition. From the conjugacy
between the lower and upper transition rate operators, and the definition of the
former, we can rewrite this requirement as saying that
0 <
Q I x i
(x i−1 ) = sup
Q∈Q
Q I x i
(x i−1 ) = sup
Q∈Q
Q(x i−1 , x i ) .
Thus, upper reachability of y, from x, requires that there exists a sequence of states
from x to y such that, at each step in this sequence, there is some transition rate
matrix Q ∈ Q which assigns strictly positive ‘speed’ of moving from the current
state in this sequence, to the next one. In other words, it should be possible for
these transitions to happen according to some of the models in our set P, but not
necessarily all, and there can be a different model allowing for this possibility at
each step. This can now be used to state the following result:
Theorem 5.7 Let P be an ICTMC and suppose that, for all x, y ∈ X , y is upper
reachable from x. Then, there is a unique lower expectation operator E P
·(X +∞ )
:
L (X ) → R such that, for all f ∈ L (X ) and all x ∈ X ,
E P
f (X +∞ )
= lim
t→+∞
E P
f (X t )
X 0 = x
= lim
t→+∞
T t f
(x) .