5 An Introduction to Imprecise Markov Chains
173
contrast, for the imprecise case, the ‘parameter’ of interest is a set Q of transition
rate matrices. Because a precise homogeneous CTMC is identified with a rate matrix
Q, it is clear that such a set Q induces a set of precise processes: simply consider
all processes for which the associated rate matrix is included in Q. However, this
induced set then only includes homogeneous Markov processes, and, as remarked
above, we aim to relax these independence assumptions. Using the parametrisation
of more general precise processes, we introduce the notion of compatibility with a
given set of rate matrices:
Definition 5.18 Let Q be a set of transition rate matrices. Then a continuous-time
stochastic process P is called compatible with Q if, for all s = s 1 , . . . , s n and
t ∈ R ≥0 such that s 1 < · · · < s n < t, and all x s ∈ X n , it holds that Q x s ,t ∈ Q,
where Q x s ,t is the time- and history-dependent rate matrix associated with P , as in
Eq. (5.8).
It can be verified that this definition includes, as a special case, the compatibility of
homogeneous CTMCs with rate matrix Q, with a given set Q, if Q ∈ Q. Similarly,
a non-homogeneous CTMC that is parametrised by a family Q t is compatible with
such a set if Q t ∈ Q for all t ∈ R ≥0 . The ICTMC P corresponding to a given set
Q, then, is taken to be the largest set of continuous-time stochastic processes that
are compatible with this Q. While perhaps not obvious, it can be proven that this set
P is then indeed a homogeneous ICTMC, in the sense that its corresponding lower
expectations satisfy the properties of Definition 5.17.
With this ICTMC in place, let us now again consider the main inferential
challenge: how to compute the corresponding lower expectation. A first attempt
could be to use Propositions 5.7 and 5.9 and optimise over Q; for some fixed
f ∈ L (X ), this would give
inf
Q∈Q
e
Qt f .
If we think about what this computes, we come to the conclusion that for each
Q ∈ Q, there is a homogeneous CTMC for which the conditional expectation of f
at time t ∈ R ≥0 is indeed e Qt f . We therefore conclude that this computes the lower
expectation with respect to all homogeneous CTMCs that are compatible with Q.
But what about the non-homogeneous and/or non-Markovian stochastic processes
that we know are also included in P? It turns out that the above expression ignores
their corresponding expectations and hence only yields an upper bound on the actual
lower expectation. In other words, we cannot use this expression to compute the
lower expectation for P.
The way to proceed is analogous to the approach in Sect. 5.3.2; we first define
a local ‘lower’ operator and then find the global lower expectation using repeated
compositions of this operator through the law of iterated lower expectation. To this
end, we associate with the set Q the corresponding lower transition rate operator
Q : L (X ) → L (X ), which is defined for all f ∈ L (X ) and all x ∈ X as
173
contrast, for the imprecise case, the ‘parameter’ of interest is a set Q of transition
rate matrices. Because a precise homogeneous CTMC is identified with a rate matrix
Q, it is clear that such a set Q induces a set of precise processes: simply consider
all processes for which the associated rate matrix is included in Q. However, this
induced set then only includes homogeneous Markov processes, and, as remarked
above, we aim to relax these independence assumptions. Using the parametrisation
of more general precise processes, we introduce the notion of compatibility with a
given set of rate matrices:
Definition 5.18 Let Q be a set of transition rate matrices. Then a continuous-time
stochastic process P is called compatible with Q if, for all s = s 1 , . . . , s n and
t ∈ R ≥0 such that s 1 < · · · < s n < t, and all x s ∈ X n , it holds that Q x s ,t ∈ Q,
where Q x s ,t is the time- and history-dependent rate matrix associated with P , as in
Eq. (5.8).
It can be verified that this definition includes, as a special case, the compatibility of
homogeneous CTMCs with rate matrix Q, with a given set Q, if Q ∈ Q. Similarly,
a non-homogeneous CTMC that is parametrised by a family Q t is compatible with
such a set if Q t ∈ Q for all t ∈ R ≥0 . The ICTMC P corresponding to a given set
Q, then, is taken to be the largest set of continuous-time stochastic processes that
are compatible with this Q. While perhaps not obvious, it can be proven that this set
P is then indeed a homogeneous ICTMC, in the sense that its corresponding lower
expectations satisfy the properties of Definition 5.17.
With this ICTMC in place, let us now again consider the main inferential
challenge: how to compute the corresponding lower expectation. A first attempt
could be to use Propositions 5.7 and 5.9 and optimise over Q; for some fixed
f ∈ L (X ), this would give
inf
Q∈Q
e
Qt f .
If we think about what this computes, we come to the conclusion that for each
Q ∈ Q, there is a homogeneous CTMC for which the conditional expectation of f
at time t ∈ R ≥0 is indeed e Qt f . We therefore conclude that this computes the lower
expectation with respect to all homogeneous CTMCs that are compatible with Q.
But what about the non-homogeneous and/or non-Markovian stochastic processes
that we know are also included in P? It turns out that the above expression ignores
their corresponding expectations and hence only yields an upper bound on the actual
lower expectation. In other words, we cannot use this expression to compute the
lower expectation for P.
The way to proceed is analogous to the approach in Sect. 5.3.2; we first define
a local ‘lower’ operator and then find the global lower expectation using repeated
compositions of this operator through the law of iterated lower expectation. To this
end, we associate with the set Q the corresponding lower transition rate operator
Q : L (X ) → L (X ), which is defined for all f ∈ L (X ) and all x ∈ X as