Chinese Named Entity Recognition with Changed BiLSTM and CRF
401
2.2 Conditional Random Filed
The Model Definition: CRF is a conditional probability distribution model of
one set of output variables with another set of input variables that are given. It
is characterized by the assumption that the output random variables constitute
Markov conditional field.
The Parametric Formalization: If P (Y |X) is linear CRF and the value of
random variable X, x is given, the value of random variable Y , y is shown as
follows:
P (y|x) =
1
Z(x)
exp
⎛
⎝
i,k
λ k t k (y i−1 , y i , x, i) +
i,l
μ l s l (y l , x, i)
⎞
⎠ ,
(10)
where t k and s l are characteristic functions which are always 0 or 1, λ k
and μ l are corresponding weights, Z(x) =
y exp
i,k λ k t k (y i−1 , y i , x, i) +
i,l μ l s l (y l , x, i)
is a normalization factor.
The parameters are learned by BFGS method, which is not described here.
The Prediction Algorithm: The CRF prediction problem is to find the output
sequence y
∗ with the maximum conditional probability given the CRF P (Y |X)
and the input sequence x. This problem is usually solved by viterbi algorithm.
Hence, the output sequence is written as:
y
∗ = arg max
w
P w (y|x) = arg max
w
exp(w · F (y, x))
Z w (x)
,
(11)
and because Z w (x) is a constant and exp(.) is a monotone increasing function.
Therefore,
y
∗ = arg max
y
(w · F (y, x))
(12)
So the CRF prediction problem is called the optimal path problem with the
largest probability of denormalization.
max
y
(w · F (y, x)),
(13)
where the path means the tag sequence and
w = (w 1 , w 2 , ..., w K )
T
,
(14)
F (y, x) = (f 1 (y, x), f 2 (y, x), ..., f K (y, x))
T
,
(15)
f k (y, x) =
n
i=1
f k (y i−1 , y i , x, i), k = 1, 2, ..., K.
(16)
To solve the optimal path, (13) can be written as:
max
y
n
i=1
w · F i (y i−1 , y i , x),
(17)
401
2.2 Conditional Random Filed
The Model Definition: CRF is a conditional probability distribution model of
one set of output variables with another set of input variables that are given. It
is characterized by the assumption that the output random variables constitute
Markov conditional field.
The Parametric Formalization: If P (Y |X) is linear CRF and the value of
random variable X, x is given, the value of random variable Y , y is shown as
follows:
P (y|x) =
1
Z(x)
exp
⎛
⎝
i,k
λ k t k (y i−1 , y i , x, i) +
i,l
μ l s l (y l , x, i)
⎞
⎠ ,
(10)
where t k and s l are characteristic functions which are always 0 or 1, λ k
and μ l are corresponding weights, Z(x) =
y exp
i,k λ k t k (y i−1 , y i , x, i) +
i,l μ l s l (y l , x, i)
is a normalization factor.
The parameters are learned by BFGS method, which is not described here.
The Prediction Algorithm: The CRF prediction problem is to find the output
sequence y
∗ with the maximum conditional probability given the CRF P (Y |X)
and the input sequence x. This problem is usually solved by viterbi algorithm.
Hence, the output sequence is written as:
y
∗ = arg max
w
P w (y|x) = arg max
w
exp(w · F (y, x))
Z w (x)
,
(11)
and because Z w (x) is a constant and exp(.) is a monotone increasing function.
Therefore,
y
∗ = arg max
y
(w · F (y, x))
(12)
So the CRF prediction problem is called the optimal path problem with the
largest probability of denormalization.
max
y
(w · F (y, x)),
(13)
where the path means the tag sequence and
w = (w 1 , w 2 , ..., w K )
T
,
(14)
F (y, x) = (f 1 (y, x), f 2 (y, x), ..., f K (y, x))
T
,
(15)
f k (y, x) =
n
i=1
f k (y i−1 , y i , x, i), k = 1, 2, ..., K.
(16)
To solve the optimal path, (13) can be written as:
max
y
n
i=1
w · F i (y i−1 , y i , x),
(17)