5.2 Various Sampling Methods
93
Using the probability vector P(t), the change in the distribution (5.53) is written as
P(t + t) = TP(t) .
(5.55)
Here the transition matrix T is
T =
⎛
⎜
⎝
a 11 · · · a 1N
. . .
. . .
. . .
a N1 · · · a NN
⎞
⎟
⎠ ,
(5.56)
in which the relation to the transition probability P (s j |s i ) is made as
a ij = P (s i |s j ))t i = j ,
(5.57)
a ii = 1 −
j =i
P (s j |s i ))t .
(5.58)
In the following, we will take = 1.
Supposing that P(0) is a certain probability vector, we assume
T
n P(0) → P eq ,
(5.59)
in the limit of n → ∞. 19 In other words, we assume that there exists a convergence
destination for the probability vector and that it is reached. In this context, the
probability distribution P eq that appears in this limit is called the equilibrium
distribution. If the equilibrium distribution P eq appears in the above limit, then it
satisfies
TP eq = P eq .
(5.60)
This is written with the matrix elements as
j
a ij P eq (s j ) = P eq (s i ) ,
(5.61)
Further dividing the sum on the left-hand side into j and i, it becomes
j =i
a ij P eq (s j ) + a ii P eq (s i ) = P eq (s i ) .
(5.62)
19 In general cases, in order for a Markov chain T whose elements are all non-negative to uniquely
converge, all transitions must satisfy the following two conditions [66]:
1. Irreducible (ergodic). That is, T is not a block diagonal.
2. Non-periodicity. That is, there is no state that always appears at a constant period.
93
Using the probability vector P(t), the change in the distribution (5.53) is written as
P(t + t) = TP(t) .
(5.55)
Here the transition matrix T is
T =
⎛
⎜
⎝
a 11 · · · a 1N
. . .
. . .
. . .
a N1 · · · a NN
⎞
⎟
⎠ ,
(5.56)
in which the relation to the transition probability P (s j |s i ) is made as
a ij = P (s i |s j ))t i = j ,
(5.57)
a ii = 1 −
j =i
P (s j |s i ))t .
(5.58)
In the following, we will take = 1.
Supposing that P(0) is a certain probability vector, we assume
T
n P(0) → P eq ,
(5.59)
in the limit of n → ∞. 19 In other words, we assume that there exists a convergence
destination for the probability vector and that it is reached. In this context, the
probability distribution P eq that appears in this limit is called the equilibrium
distribution. If the equilibrium distribution P eq appears in the above limit, then it
satisfies
TP eq = P eq .
(5.60)
This is written with the matrix elements as
j
a ij P eq (s j ) = P eq (s i ) ,
(5.61)
Further dividing the sum on the left-hand side into j and i, it becomes
j =i
a ij P eq (s j ) + a ii P eq (s i ) = P eq (s i ) .
(5.62)
19 In general cases, in order for a Markov chain T whose elements are all non-negative to uniquely
converge, all transitions must satisfy the following two conditions [66]:
1. Irreducible (ergodic). That is, T is not a block diagonal.
2. Non-periodicity. That is, there is no state that always appears at a constant period.