96
5 Sampling
In the next section, we introduce concrete Markov chain Monte Carlo algorithms.
5.3 Sampling Method with the Detailed Balance
In this section, as sampling methods that satisfy the detailed balance, we introduce
the (narrowly defined) Metropolis method (algorithm) [70] and the heatbath method
(algorithm) [71] (which is the Gibbs sampler in the machine learning context [72]).
The method introduced here is generally called the Metropolis-Hastings algorithm
[73]. Since these are derived from the principle of detailed balance described in the
previous section, they converge to the target probability distribution. 24
5.3.1 Metropolis Method
Here, we introduce the Metropolis method in a narrow sense, as a concrete method
for how to take the transition probability P (s i |s j ). As a feature of the Metropolis
method, there is no need to explicitly calculate the probability distribution, and it is
sufficient if there is only a Hamiltonian (energy function) in physics.
To make the discussion concrete, consider the equilibrium distribution P eq (s j ) as
follows:
P eq (s i ) =
1
Z β
e
−βH [s i ] .
(5.69)
Here, s i is a state, and H [s i ] is the Hamiltonian of the system.
Now, prepare two states s i and s j such that H [s i ] < H [s j ]. The state s i has lower
energy than the state s j , so it is plausible that s i is realized with high probability.
From this consideration, let us take the transition probabilities as follows:
P (s i |s j ) = 1 .
(5.70)
Then, the transition probability of the opposite direction P (s j |s i ) is determined from
the principle of detailed balance,
e
−βH [s j ]
= P (s j |s i )e
−βH [s i ] .
(5.71)
Namely, we get
P (s j |s i ) = e
−β(H [s j ]−H [s i ]) .
(5.72)
24 We will not introduce global updates such as cluster algorithms, exchange Monte Carlo, or
Hamiltonian (or hybrid) Monte Carlo (HMC) used in the context of Bayesian statistics. Interested
readers should also learn about these.
5 Sampling
In the next section, we introduce concrete Markov chain Monte Carlo algorithms.
5.3 Sampling Method with the Detailed Balance
In this section, as sampling methods that satisfy the detailed balance, we introduce
the (narrowly defined) Metropolis method (algorithm) [70] and the heatbath method
(algorithm) [71] (which is the Gibbs sampler in the machine learning context [72]).
The method introduced here is generally called the Metropolis-Hastings algorithm
[73]. Since these are derived from the principle of detailed balance described in the
previous section, they converge to the target probability distribution. 24
5.3.1 Metropolis Method
Here, we introduce the Metropolis method in a narrow sense, as a concrete method
for how to take the transition probability P (s i |s j ). As a feature of the Metropolis
method, there is no need to explicitly calculate the probability distribution, and it is
sufficient if there is only a Hamiltonian (energy function) in physics.
To make the discussion concrete, consider the equilibrium distribution P eq (s j ) as
follows:
P eq (s i ) =
1
Z β
e
−βH [s i ] .
(5.69)
Here, s i is a state, and H [s i ] is the Hamiltonian of the system.
Now, prepare two states s i and s j such that H [s i ] < H [s j ]. The state s i has lower
energy than the state s j , so it is plausible that s i is realized with high probability.
From this consideration, let us take the transition probabilities as follows:
P (s i |s j ) = 1 .
(5.70)
Then, the transition probability of the opposite direction P (s j |s i ) is determined from
the principle of detailed balance,
e
−βH [s j ]
= P (s j |s i )e
−βH [s i ] .
(5.71)
Namely, we get
P (s j |s i ) = e
−β(H [s j ]−H [s i ]) .
(5.72)
24 We will not introduce global updates such as cluster algorithms, exchange Monte Carlo, or
Hamiltonian (or hybrid) Monte Carlo (HMC) used in the context of Bayesian statistics. Interested
readers should also learn about these.