170
T. Krak
We therefore see that the matrix Q can be used to approximately compute the
transition probabilities over a small enough time step.
An obvious next question is if we can extrapolate this to compute the matrix T t
that contains the transition probabilities over an arbitrary duration t. Indeed we can,
although it requires a bit of setup. For any t ∈ R ≥0 , first define the transition matrix
of the CTMC after time t:
T t (x, y) := P (X t = y | X 0 = x)
for all x, y ∈ X .
Then we differentiate in t; to this end, first fix Δ > 0, and use the Markov property
and homogeneity to derive that T t+Δ = T t T Δ = T Δ T t (c.f. Proposition 5.2). Then
we proceed by using Eq. (5.6):
d T t
d t
= lim
Δ→0 +
T t+Δ − T t
Δ
= lim
Δ→0 +
T Δ T t − T t
Δ
=
lim
Δ→0 +
T Δ − I
Δ
T t = QT t .
Using also Eq. (5.5), we can now write the matrix differential equation
d T t
d t
= QT t ,
T 0 = I ,
whose solution is the matrix exponential of Qt:
T t = e
Qt .
We recall from Proposition 5.4 that the dynamic behaviour of a homogeneous
discrete-time Markov chain can be characterised by a single transition matrix T and
that therefore this matrix constitutes the canonical parameter of the process. Because
the matrix Q can be used to (re-)construct the transition matrices of a homogeneous
CTMC over any time duration, it plays the same role here.
Proposition 5.7 Let {X t } t∈R ≥0 be a continuous-time homogeneous Markov chain,
with transition rate matrix Q as defined above. Then for all t ∈ R ≥0 , the transition
probabilities P (X t = y | X 0 = x), x, y ∈ X after time t are given by the elements
T t (x, y) of the transition matrix T t = e Qt .
While we do not aim to give a complete treatment on the interpretation of the
matrix exponential, some properties are worth pointing out. First of all, it can be
defined analogously to the exponential function of real numbers, that is, through a
Taylor expansion around zero. Specifically, it holds that
T t = e
Qt
:=
+∞
k=0
t k Q k
k!
.
Thus, the approximation in Eq. (5.7) can be seen as a first-order truncation of the
series above.
T. Krak
We therefore see that the matrix Q can be used to approximately compute the
transition probabilities over a small enough time step.
An obvious next question is if we can extrapolate this to compute the matrix T t
that contains the transition probabilities over an arbitrary duration t. Indeed we can,
although it requires a bit of setup. For any t ∈ R ≥0 , first define the transition matrix
of the CTMC after time t:
T t (x, y) := P (X t = y | X 0 = x)
for all x, y ∈ X .
Then we differentiate in t; to this end, first fix Δ > 0, and use the Markov property
and homogeneity to derive that T t+Δ = T t T Δ = T Δ T t (c.f. Proposition 5.2). Then
we proceed by using Eq. (5.6):
d T t
d t
= lim
Δ→0 +
T t+Δ − T t
Δ
= lim
Δ→0 +
T Δ T t − T t
Δ
=
lim
Δ→0 +
T Δ − I
Δ
T t = QT t .
Using also Eq. (5.5), we can now write the matrix differential equation
d T t
d t
= QT t ,
T 0 = I ,
whose solution is the matrix exponential of Qt:
T t = e
Qt .
We recall from Proposition 5.4 that the dynamic behaviour of a homogeneous
discrete-time Markov chain can be characterised by a single transition matrix T and
that therefore this matrix constitutes the canonical parameter of the process. Because
the matrix Q can be used to (re-)construct the transition matrices of a homogeneous
CTMC over any time duration, it plays the same role here.
Proposition 5.7 Let {X t } t∈R ≥0 be a continuous-time homogeneous Markov chain,
with transition rate matrix Q as defined above. Then for all t ∈ R ≥0 , the transition
probabilities P (X t = y | X 0 = x), x, y ∈ X after time t are given by the elements
T t (x, y) of the transition matrix T t = e Qt .
While we do not aim to give a complete treatment on the interpretation of the
matrix exponential, some properties are worth pointing out. First of all, it can be
defined analogously to the exponential function of real numbers, that is, through a
Taylor expansion around zero. Specifically, it holds that
T t = e
Qt
:=
+∞
k=0
t k Q k
k!
.
Thus, the approximation in Eq. (5.7) can be seen as a first-order truncation of the
series above.