Chinese Named Entity Recognition with Changed BiLSTM and CRF
399
and a better accuracy [9]. But their method did not consider the reverse meaning
of texts. Also, Zheng S, et al. used a BiLSTM network to deal with this problem
and a LSTM network to replace CRF to gain the deeper semantic [10], but they
did not consider the probabilistic relationship.
To solve these problems, we propose an advanced BiLSTM network to achieve
better deep semantics and probability relationships. We use two LSTM to train
a sequence and return a sequence. What is more, we use two reverse LSTM to
train the same sequence and return a sequence too. Then, we concatenate the
two sequences to one sequence. Hence, we can obtain deep semantics and the
relationships including the forward meaning and backward meaning together.
Finally, we use a CRF to obtain the probabilistic relation.
In this paper, in Sect. 1, we introduce the background and the overall idea.
In Sect. 2 , we introduce the important models in our algorithm. In Sect. 3 , we
give out our experiment results. In Sect. 4, a conclusion is given.
2 Mathematic Models
2.1 Improved BiLSTM
The Disadvantage of RNN: Recurrent neural networks (RNN) are implemented by reusing a cell structure. Hence, the output of the current moment is
related to the past.
The expression function of RNN is:
h t = f (W · x t + U · h t−1 + b),
(1)
y t = V · h t ,
(2)
where x t and y t are the input and output at time t, respectively. h t is the memory
information at time t, and f (z) is an activation function, which is usually a tanh
function. W, U, b, v are the parameters of the network.
Because RNN is implemented by reusing a cell structure, we just train
W, U, b, v by iterations and do not need other parameters.
However, there is a disadvantage of RNN. It can not remember long time
information because of gradient diffusion. The appearance of LSTM solved this
problem.
The Advantage of LSTM: LSTM is a variant of RNN, which can effectively
solve the gradient diffusion problem of simple RNN. It mainly improves the
following two parts.
First, it adds a new internal state c t and retains the original external state h t .
Hence, gradient diffusion is suppressed by combining linearity and nonlinearity.
Second, it controls the amount of information transmitted through three gates,
ensuring that the linear transmission does not lead to too much information.
Therefore, its formulas are written as:
c t = f t c t−1 + i t ˆ
c t ,
(3)
399
and a better accuracy [9]. But their method did not consider the reverse meaning
of texts. Also, Zheng S, et al. used a BiLSTM network to deal with this problem
and a LSTM network to replace CRF to gain the deeper semantic [10], but they
did not consider the probabilistic relationship.
To solve these problems, we propose an advanced BiLSTM network to achieve
better deep semantics and probability relationships. We use two LSTM to train
a sequence and return a sequence. What is more, we use two reverse LSTM to
train the same sequence and return a sequence too. Then, we concatenate the
two sequences to one sequence. Hence, we can obtain deep semantics and the
relationships including the forward meaning and backward meaning together.
Finally, we use a CRF to obtain the probabilistic relation.
In this paper, in Sect. 1, we introduce the background and the overall idea.
In Sect. 2 , we introduce the important models in our algorithm. In Sect. 3 , we
give out our experiment results. In Sect. 4, a conclusion is given.
2 Mathematic Models
2.1 Improved BiLSTM
The Disadvantage of RNN: Recurrent neural networks (RNN) are implemented by reusing a cell structure. Hence, the output of the current moment is
related to the past.
The expression function of RNN is:
h t = f (W · x t + U · h t−1 + b),
(1)
y t = V · h t ,
(2)
where x t and y t are the input and output at time t, respectively. h t is the memory
information at time t, and f (z) is an activation function, which is usually a tanh
function. W, U, b, v are the parameters of the network.
Because RNN is implemented by reusing a cell structure, we just train
W, U, b, v by iterations and do not need other parameters.
However, there is a disadvantage of RNN. It can not remember long time
information because of gradient diffusion. The appearance of LSTM solved this
problem.
The Advantage of LSTM: LSTM is a variant of RNN, which can effectively
solve the gradient diffusion problem of simple RNN. It mainly improves the
following two parts.
First, it adds a new internal state c t and retains the original external state h t .
Hence, gradient diffusion is suppressed by combining linearity and nonlinearity.
Second, it controls the amount of information transmitted through three gates,
ensuring that the linear transmission does not lead to too much information.
Therefore, its formulas are written as:
c t = f t c t−1 + i t ˆ
c t ,
(3)