5 An Introduction to Imprecise Markov Chains
163
Exactly analogous to the precise case, by repeatedly pulling back until we reach the
root of the tree, we eventually compute
E P [f (X 0:n )] = inf
p ∈P
x∈X
p (x)E P
f (X 0:n )
X 0 = x
,
which is the lower expectation of interest.
As before, the need to specify these (imprecise) local models P w for all
situations w ∈ X ∗
makes such a model difficult to work with. This is simplified for
imprecise Markov chains; note that we here assume the analogue of homogeneity to
hold implicitly:
Definition 5.13 (Homogeneous IDTMC as imprecise probability tree) An
imprecise probability tree (X ∗
, ≺, P (·) ) is called an imprecise homogeneous
discrete-time Markov chain if P v = P w for all v, w ∈ X ∗ for which v = w .
Corollary 5.5 Let (X ∗
, ≺, P (·) ) be a homogeneous IDTMC. Then P w = P x for
all x ∈ X and all w ∈ X ∗ such that w = x.
Proof Trivial from Definition 5.13 and the fact that all x ∈ X are also situations.
As above, an IDTMC (X ∗
, ≺, P (·) ) has a set of compatible precise probability
trees, each of which induces a measure P , and these are collected in the set P, which
is the measure-theoretic IDTMC representation from Definition 5.10. Observe that a
precise probability tree does not have to be a (homogeneous) Markov chain, for it to
be compatible with a given IDTMC! That is, to be compatible, each local model p w ,
w ∈ X ∗
, should be in the set P w , and this set depends only on the most recent
state w of the situation w. But, while in a different situation v such that v = w ,
we do require that p v ∈ P v = P w ; we do not require that p v = p w !
We will next illustrate that the law of iterated lower expectation simplifies
further for imprecise Markov chains. We do this again by considering the imprecise
counterpart of Bayesian networks.
5.3.2 Credal Networks
We here consider the graphical representation of imprecise Markov chains as
credal networks. This is the imprecise generalisation of the Bayesian network
representation that we encountered in Sect. 5.2.2. The graphical structure is as
before, with the notable differences being (i) the local models (which are here
replaced with imprecise local models) and (ii) the interpretation of the independence
properties induced by the arcs. Regarding the second point, it suffices for our present
purpose to note that we interpret the structure as a credal network under epistemic
irrelevance. This then has the same consequence as that stated in the beginning of
Sect. 5.3: given the value of the parent of a node X t , t ∈ N 0 , the lower expectation
163
Exactly analogous to the precise case, by repeatedly pulling back until we reach the
root of the tree, we eventually compute
E P [f (X 0:n )] = inf
p ∈P
x∈X
p (x)E P
f (X 0:n )
X 0 = x
,
which is the lower expectation of interest.
As before, the need to specify these (imprecise) local models P w for all
situations w ∈ X ∗
makes such a model difficult to work with. This is simplified for
imprecise Markov chains; note that we here assume the analogue of homogeneity to
hold implicitly:
Definition 5.13 (Homogeneous IDTMC as imprecise probability tree) An
imprecise probability tree (X ∗
, ≺, P (·) ) is called an imprecise homogeneous
discrete-time Markov chain if P v = P w for all v, w ∈ X ∗ for which v = w .
Corollary 5.5 Let (X ∗
, ≺, P (·) ) be a homogeneous IDTMC. Then P w = P x for
all x ∈ X and all w ∈ X ∗ such that w = x.
Proof Trivial from Definition 5.13 and the fact that all x ∈ X are also situations.
As above, an IDTMC (X ∗
, ≺, P (·) ) has a set of compatible precise probability
trees, each of which induces a measure P , and these are collected in the set P, which
is the measure-theoretic IDTMC representation from Definition 5.10. Observe that a
precise probability tree does not have to be a (homogeneous) Markov chain, for it to
be compatible with a given IDTMC! That is, to be compatible, each local model p w ,
w ∈ X ∗
, should be in the set P w , and this set depends only on the most recent
state w of the situation w. But, while in a different situation v such that v = w ,
we do require that p v ∈ P v = P w ; we do not require that p v = p w !
We will next illustrate that the law of iterated lower expectation simplifies
further for imprecise Markov chains. We do this again by considering the imprecise
counterpart of Bayesian networks.
5.3.2 Credal Networks
We here consider the graphical representation of imprecise Markov chains as
credal networks. This is the imprecise generalisation of the Bayesian network
representation that we encountered in Sect. 5.2.2. The graphical structure is as
before, with the notable differences being (i) the local models (which are here
replaced with imprecise local models) and (ii) the interpretation of the independence
properties induced by the arcs. Regarding the second point, it suffices for our present
purpose to note that we interpret the structure as a credal network under epistemic
irrelevance. This then has the same consequence as that stated in the beginning of
Sect. 5.3: given the value of the parent of a node X t , t ∈ N 0 , the lower expectation