5 An Introduction to Imprecise Markov Chains
169
dimension. And, for each such positive Δ, we can associate a discrete-time Markov
chain and use all the previous interpretations that we developed.
We first remark that the naive limit does not encode a lot of information; ignoring
possible issues of continuity, it trivially holds that
lim
Δ→0 +
P (X Δ = y | X 0 = x) = P (X 0 = y | X 0 = x) =
1 if y = x, and
0 otherwise.
(5.5)
In matrix notation this reads as lim Δ→0 + T Δ = I , where I denotes the |X | × |X |
identity matrix. Colloquially, we might understand this as saying that ‘if time does
not evolve, the system does not change’. This is clearly an almost tautological
statement to make of what may be interpreted as a dynamical system. So let us
consider how the system does change as time evolves. The natural representation
for this is obviously the derivative of the transition matrix T Δ ; this is the limit
interpretation that we shall use. Ignoring technical issues of differentiability, we
have
d T Δ
d Δ
Δ=0
= lim
Δ→0 +
T Δ − I
Δ
=: Q ,
(5.6)
where we have used the previous observation that T 0 = I . On the right-hand
side, the term Q is called the transition rate matrix of the homogeneous CTMC
(or sometimes simply the rate matrix). It is clear from the above definition that it
encodes the rate of change of the transition probabilities around time zero. It satisfies
the following properties:
Definition 5.16 (Transition Rate Matrix) A real-valued |X | × |X | matrix Q is
called a transition rate matrix if, for all x ∈ X , it holds that
1. Q(x, y) ≥ 0 for all y ∈ X such that x = y and
2.
y∈X Q(x, y) = 0.
The elements Q(x, y) of a rate matrix can be interpreted as the ‘speed’ with which
the process moves from the state x to the state y. In the above definition, the two
conditions imply that the diagonal elements Q(x, x) are always non-positive. On the
other hand, the first condition states that the off-diagonal elements are non-negative.
Combined this can be understood as saying that the system will move ‘out’ of the
current state (the non-positivity of the diagonal elements) and ‘into’ some other
states (the non-negativity of the off-diagonals).
A more concrete way to interpret the rate-matrix is through a linearised approximation of the transition probabilities over a small enough time step. That is, it
follows from Eq. (5.6) that, for ‘small enough’ Δ > 0, it holds that Q ≈ (T Δ −I ) 1 /Δ;
hence also
T Δ ≈ I + ΔQ .
(5.7)
169
dimension. And, for each such positive Δ, we can associate a discrete-time Markov
chain and use all the previous interpretations that we developed.
We first remark that the naive limit does not encode a lot of information; ignoring
possible issues of continuity, it trivially holds that
lim
Δ→0 +
P (X Δ = y | X 0 = x) = P (X 0 = y | X 0 = x) =
1 if y = x, and
0 otherwise.
(5.5)
In matrix notation this reads as lim Δ→0 + T Δ = I , where I denotes the |X | × |X |
identity matrix. Colloquially, we might understand this as saying that ‘if time does
not evolve, the system does not change’. This is clearly an almost tautological
statement to make of what may be interpreted as a dynamical system. So let us
consider how the system does change as time evolves. The natural representation
for this is obviously the derivative of the transition matrix T Δ ; this is the limit
interpretation that we shall use. Ignoring technical issues of differentiability, we
have
d T Δ
d Δ
Δ=0
= lim
Δ→0 +
T Δ − I
Δ
=: Q ,
(5.6)
where we have used the previous observation that T 0 = I . On the right-hand
side, the term Q is called the transition rate matrix of the homogeneous CTMC
(or sometimes simply the rate matrix). It is clear from the above definition that it
encodes the rate of change of the transition probabilities around time zero. It satisfies
the following properties:
Definition 5.16 (Transition Rate Matrix) A real-valued |X | × |X | matrix Q is
called a transition rate matrix if, for all x ∈ X , it holds that
1. Q(x, y) ≥ 0 for all y ∈ X such that x = y and
2.
y∈X Q(x, y) = 0.
The elements Q(x, y) of a rate matrix can be interpreted as the ‘speed’ with which
the process moves from the state x to the state y. In the above definition, the two
conditions imply that the diagonal elements Q(x, x) are always non-positive. On the
other hand, the first condition states that the off-diagonal elements are non-negative.
Combined this can be understood as saying that the system will move ‘out’ of the
current state (the non-positivity of the diagonal elements) and ‘into’ some other
states (the non-negativity of the off-diagonals).
A more concrete way to interpret the rate-matrix is through a linearised approximation of the transition probabilities over a small enough time step. That is, it
follows from Eq. (5.6) that, for ‘small enough’ Δ > 0, it holds that Q ≈ (T Δ −I ) 1 /Δ;
hence also
T Δ ≈ I + ΔQ .
(5.7)