54
3 Basics of Neural Networks
Here, k B is an important constant called the Boltzmann constant, and Z is called
the partition function, defined as
Z =
d
e
−
H (d)
k B T .
(3.93)
In the main text, we put k B T = 1.
Simple example: law of equipartition of energy
As an example, consider d as the position x and momentum p of a particle in a box
of length L, with the standard kinetic energy function as its energy. The expectation
value of energy is
=
L
0
dx
+∞
−∞
dp
p 2
2m
e
−
p 2
2mk B T
Z
.
(3.94)
Here
Z =
L
0
dx
+∞
−∞
dp e
−
p 2
2mkT = L(2πmk B T )
1/2 .
(3.95)
The calculation of looks a bit difficult, but using β =
1
k B T we find
Z = L
2πm
β
1/2
,
(3.96)
= −
∂
∂β
log Z =
1
2
1
β
=
1
2
k B T .
(3.97)
In other words, when the system temperature T is high (i.e. when the system is
hot), the expectation value of energy is high, and when the temperature is low,
the expectation value of energy is low. This is consistent with our intuition. In
addition, if we consider the case of three spatial dimensions, we can obtain the
famous formula
3
2 k B T as the expectation value of energy.
Bracket Notation in Quantum Mechanics
In the derivation of the backpropagation method, bracket notation has been
introduced as a simple method. This is nothing more than just writing a vector as a
ket,
v = |v .
(3.98)