176
T. Krak
5.5 Literature and Further Reading
Let us conclude this chapter by providing pointers to the literature on which the
material in this chapter is based. We will also briefly discuss some parts of the
literature that are related but not quite the same as what we covered here.
First of all, there exists an extensive body of literature on (precise) Markov
chains, both in discrete- and in continuous times. It would be nigh impossible to
give a complete overview here, but we think that [1, 38] make excellent introductory
reads. For a broad and general introduction to the theory for imprecise probability,
which lies at the heart of the models that we discussed here, we refer the reader
to [3, 50]. The difference between the notions of strong independence and epistemic
irrelevance—which we have stressed repeatedly and which is a crucial property of
imprecise Markov chains as we treated them here—is discussed, e.g. in [5, 37].
For the interpretation of Markov chains using probability trees, see, for example, [13, 18, 32]. This interpretation is also closely related to the game-theoretic
formalisation of probabilities using the theory of martingales (which we did not
cover here). The interested reader may want to pursue [13, 32, 49].
For an account of the general theory of Bayesian networks, see [39]. For their
imprecise generalisation—credal networks—references [2, 6, 7, 9, 11] discuss a lot
of the general theory.
Imprecise discrete-time Markov chains are discussed, e.g. in [15, 17, 27]. For
imprecise continuous-time Markov chains, see [30, 44]. A treatment of the matrix
exponential, which is crucial to computational methods for CTMCs, is given in [48].
Reference [19] discusses the current state-of-the-art to efficiently compute the
imprecise generalisation of the matrix exponential, which we have seen is crucial
for computing inferences in ICTMCs.
Detailed treatments on the long-term (limit) behaviour in IDTMCs can be found
in [14, 16, 26, 45]. Reference [10] provides the necessary and sufficient conditions
for the limit behaviour of ICTMCs, and [19] also discusses computational methods
to numerically approximate this limit. We remark that Theorems 5.4 and 5.7 in this
chapter are stated in a simplified form compared to their statement in the literature.
In particular, the results in [10, 14] are stronger; for instance, [10] in fact provides
necessary and sufficient conditions for the convergence of an ICTMC, whereas
Theorem 5.7 only states a sufficient condition.
Some examples of the merits of imprecise Markov chains in applications are
provided by [40, 46, 47]. A domain for which the applicability of (imprecise)
Markov chains has been extensively studied, is queueing theory [8, 33–36].
A generalisation of Markov chains that we have not discussed, but which is
nevertheless important in many practical applications, is hidden Markov chains.
There, the stochastic process cannot be observed directly, but only through a noisy
measurement model. Their imprecise treatment is discussed, e.g. in [4, 12, 31].
Fields that are closely related to the theory of imprecise Markov chains are
controlled Markov processes [21] and Markov decision processes [22, 28, 41, 51].
There also, the process under study has its parameters changed over time. However,
T. Krak
5.5 Literature and Further Reading
Let us conclude this chapter by providing pointers to the literature on which the
material in this chapter is based. We will also briefly discuss some parts of the
literature that are related but not quite the same as what we covered here.
First of all, there exists an extensive body of literature on (precise) Markov
chains, both in discrete- and in continuous times. It would be nigh impossible to
give a complete overview here, but we think that [1, 38] make excellent introductory
reads. For a broad and general introduction to the theory for imprecise probability,
which lies at the heart of the models that we discussed here, we refer the reader
to [3, 50]. The difference between the notions of strong independence and epistemic
irrelevance—which we have stressed repeatedly and which is a crucial property of
imprecise Markov chains as we treated them here—is discussed, e.g. in [5, 37].
For the interpretation of Markov chains using probability trees, see, for example, [13, 18, 32]. This interpretation is also closely related to the game-theoretic
formalisation of probabilities using the theory of martingales (which we did not
cover here). The interested reader may want to pursue [13, 32, 49].
For an account of the general theory of Bayesian networks, see [39]. For their
imprecise generalisation—credal networks—references [2, 6, 7, 9, 11] discuss a lot
of the general theory.
Imprecise discrete-time Markov chains are discussed, e.g. in [15, 17, 27]. For
imprecise continuous-time Markov chains, see [30, 44]. A treatment of the matrix
exponential, which is crucial to computational methods for CTMCs, is given in [48].
Reference [19] discusses the current state-of-the-art to efficiently compute the
imprecise generalisation of the matrix exponential, which we have seen is crucial
for computing inferences in ICTMCs.
Detailed treatments on the long-term (limit) behaviour in IDTMCs can be found
in [14, 16, 26, 45]. Reference [10] provides the necessary and sufficient conditions
for the limit behaviour of ICTMCs, and [19] also discusses computational methods
to numerically approximate this limit. We remark that Theorems 5.4 and 5.7 in this
chapter are stated in a simplified form compared to their statement in the literature.
In particular, the results in [10, 14] are stronger; for instance, [10] in fact provides
necessary and sufficient conditions for the convergence of an ICTMC, whereas
Theorem 5.7 only states a sufficient condition.
Some examples of the merits of imprecise Markov chains in applications are
provided by [40, 46, 47]. A domain for which the applicability of (imprecise)
Markov chains has been extensively studied, is queueing theory [8, 33–36].
A generalisation of Markov chains that we have not discussed, but which is
nevertheless important in many practical applications, is hidden Markov chains.
There, the stochastic process cannot be observed directly, but only through a noisy
measurement model. Their imprecise treatment is discussed, e.g. in [4, 12, 31].
Fields that are closely related to the theory of imprecise Markov chains are
controlled Markov processes [21] and Markov decision processes [22, 28, 41, 51].
There also, the process under study has its parameters changed over time. However,