5 An Introduction to Imprecise Markov Chains
145
In what follows, we will be interested in computing the expectation of some realvalued function, whose value depends on the specific realisation of the stochastic
process. To prevent technical difficulties, we will assume that this function only
depends on a finite number of time points; without loss of generality, we can then
assume that it is a map f : X n → R, with n ∈ N, whose value depends on the
n random variables X t , with t = t 1 , . . . , t n in T. We collect in the set L (X n ) all
such real-valued functions on X n . The expected value of any such f ∈ L (X n )
on the n time points t is defined as
E P
f (X t )
:=
x t ∈X n
f (x t )P (X t = x t ) ,
(5.1)
where we have implicitly introduced the intuitive notation for the set
(X t = x t ) :=
ω ∈ Ω :
∀i ∈ {1, . . . , n} : ω(t i ) = x t i
.
In Eq. (5.1), we use the subscript P for the expectation operator E P to make explicit
that it is taken with respect to the measure P ; this will be notationally convenient
further on.
We finish this first introduction by recalling the notion of conditional probabilities and conditional expectations. For any two finite sequences of time points t and
s in T, the conditional probability of X t , given X s , is derived using Bayes’ rule:
P (X t | X s ) :=
P (X s , X t )
P (X s )
,
whenever P (X s ) is strictly positive. The necessity of the final condition is obvious;
it leads to a division by zero whenever it does not hold.
Using this notion of conditional probability, we can define conditional expectations analogously. Suppose the sequences s and t are of length n, m ∈ N,
respectively. Then for any f ∈ L (X n+m ) on X s , X t we define, for all x s ∈ X n ,
E P
f (X s , X t )
X s = x s
:=
x t ∈X m
f (x s , x t )P (X t = x t | X s = x s ) .
5.2.1 Probability Trees
The preceding discussion introduced stochastic processes in a very general, but
rather abstract sense. We will build further intuition by next offering a different view
and representation, by means of probability trees. In the remainder of this section,
unless otherwise specified, we will focus on discrete-time stochastic processes,
whence we identify T = N 0 .
145
In what follows, we will be interested in computing the expectation of some realvalued function, whose value depends on the specific realisation of the stochastic
process. To prevent technical difficulties, we will assume that this function only
depends on a finite number of time points; without loss of generality, we can then
assume that it is a map f : X n → R, with n ∈ N, whose value depends on the
n random variables X t , with t = t 1 , . . . , t n in T. We collect in the set L (X n ) all
such real-valued functions on X n . The expected value of any such f ∈ L (X n )
on the n time points t is defined as
E P
f (X t )
:=
x t ∈X n
f (x t )P (X t = x t ) ,
(5.1)
where we have implicitly introduced the intuitive notation for the set
(X t = x t ) :=
ω ∈ Ω :
∀i ∈ {1, . . . , n} : ω(t i ) = x t i
.
In Eq. (5.1), we use the subscript P for the expectation operator E P to make explicit
that it is taken with respect to the measure P ; this will be notationally convenient
further on.
We finish this first introduction by recalling the notion of conditional probabilities and conditional expectations. For any two finite sequences of time points t and
s in T, the conditional probability of X t , given X s , is derived using Bayes’ rule:
P (X t | X s ) :=
P (X s , X t )
P (X s )
,
whenever P (X s ) is strictly positive. The necessity of the final condition is obvious;
it leads to a division by zero whenever it does not hold.
Using this notion of conditional probability, we can define conditional expectations analogously. Suppose the sequences s and t are of length n, m ∈ N,
respectively. Then for any f ∈ L (X n+m ) on X s , X t we define, for all x s ∈ X n ,
E P
f (X s , X t )
X s = x s
:=
x t ∈X m
f (x s , x t )P (X t = x t | X s = x s ) .
5.2.1 Probability Trees
The preceding discussion introduced stochastic processes in a very general, but
rather abstract sense. We will build further intuition by next offering a different view
and representation, by means of probability trees. In the remainder of this section,
unless otherwise specified, we will focus on discrete-time stochastic processes,
whence we identify T = N 0 .