168
T. Krak
For instance, we could try to draw a ‘continuous-time’ probability tree, where the
local model of a situation with terminal state w is given by a probability mass
function P (X t | X 0 = w ). But what is the time t that we should use? When
we were working in discrete-time, the approach was to use the next time point,
as viewed from the current situation. But of course, there is no ‘next’ time t when
working in continuous-time! This difficulty of using graphical representations is
the main reason that we have postponed the treatment of continuous-time processes
until now, thereby hopefully allowing the reader to first develop some graphical
intuition for the discrete-time case.
Nevertheless, all is not lost; the first interpretation that we will consider is to
view continuous-time processes as limits of discrete-time ones. To this end, it will
be convenient to consider the transition-matrix T associated with a homogeneous
DTMC. Let us recall from Sects. 5.2.2 and 5.2.3 that the elements of such a matrix
represent the ‘transition probabilities’ of the system, that is, the probability of
moving from a state x to a state y, in one time step:
T (x, y) = P (X 1 = y | X 0 = x) .
We can use this formalism to interpret the continuous-time case, by simply ‘fixing
the length of the step’. That is, consider some ‘step size’ Δ > 0. Then, for a
homogeneous CTMC, we known that
P (X t+Δ | X t ) = P (X Δ | X 0 ) ,
for all t ∈ R ≥0 , so we can collect these ‘transition probabilities’ in a matrix T Δ :
T Δ (x, y) := P (X Δ = y | X 0 = x)
for all x, y ∈ X .
Clearly, the elements of T Δ are the probabilities for the system to end up in a state y,
if it is currently in a state x, after a time duration of Δ has elapsed. Provided, then,
that we are not interested in a granularity of the time-dimension that is finer than Δ,
this representation suffices. The matrix T Δ can be associated with a DTMC, and all
the previous results can be used. For instance, for any multiple n ∈ N of Δ, we use
Proposition 5.2 to find that
P (X nΔ = y | X 0 = x) = T
n
Δ (x, y) .
But, of course, the point of using the continuous-time representation is that we
are interested in an arbitrarily fine granularity of the time-dimension. In particular,
the measure-theoretic definition encodes this arbitrary granularity, and it seems a
waste to only focus on the restriction to a single step size Δ. The ‘trick’, then, is
to take the limit as Δ goes to zero, and somehow usefully represent this limit. It is
hopefully clear from the above discussion that, as we decrease Δ further and further,
the associated transition matrix T Δ covers increasingly smaller steps along the time-
T. Krak
For instance, we could try to draw a ‘continuous-time’ probability tree, where the
local model of a situation with terminal state w is given by a probability mass
function P (X t | X 0 = w ). But what is the time t that we should use? When
we were working in discrete-time, the approach was to use the next time point,
as viewed from the current situation. But of course, there is no ‘next’ time t when
working in continuous-time! This difficulty of using graphical representations is
the main reason that we have postponed the treatment of continuous-time processes
until now, thereby hopefully allowing the reader to first develop some graphical
intuition for the discrete-time case.
Nevertheless, all is not lost; the first interpretation that we will consider is to
view continuous-time processes as limits of discrete-time ones. To this end, it will
be convenient to consider the transition-matrix T associated with a homogeneous
DTMC. Let us recall from Sects. 5.2.2 and 5.2.3 that the elements of such a matrix
represent the ‘transition probabilities’ of the system, that is, the probability of
moving from a state x to a state y, in one time step:
T (x, y) = P (X 1 = y | X 0 = x) .
We can use this formalism to interpret the continuous-time case, by simply ‘fixing
the length of the step’. That is, consider some ‘step size’ Δ > 0. Then, for a
homogeneous CTMC, we known that
P (X t+Δ | X t ) = P (X Δ | X 0 ) ,
for all t ∈ R ≥0 , so we can collect these ‘transition probabilities’ in a matrix T Δ :
T Δ (x, y) := P (X Δ = y | X 0 = x)
for all x, y ∈ X .
Clearly, the elements of T Δ are the probabilities for the system to end up in a state y,
if it is currently in a state x, after a time duration of Δ has elapsed. Provided, then,
that we are not interested in a granularity of the time-dimension that is finer than Δ,
this representation suffices. The matrix T Δ can be associated with a DTMC, and all
the previous results can be used. For instance, for any multiple n ∈ N of Δ, we use
Proposition 5.2 to find that
P (X nΔ = y | X 0 = x) = T
n
Δ (x, y) .
But, of course, the point of using the continuous-time representation is that we
are interested in an arbitrarily fine granularity of the time-dimension. In particular,
the measure-theoretic definition encodes this arbitrary granularity, and it seems a
waste to only focus on the restriction to a single step size Δ. The ‘trick’, then, is
to take the limit as Δ goes to zero, and somehow usefully represent this limit. It is
hopefully clear from the above discussion that, as we decrease Δ further and further,
the associated transition matrix T Δ covers increasingly smaller steps along the time-