74
Q. Wu et al.
4.2 Performance Evaluation
We repeat these experiences in three models above-mentioned and get Fig. 6
which show the F1 score of the performance of three models under different
SINR. F 1 score is calculated as F 1 score =
2P R
P +R (where P represents precision and
R represents recall). Obviously, when using the noiseless signal as label, we get
the best performance, especially at low SINR. In the case of noise label, although
the MAE and MAPE values of Bi-LSTM are smaller than those of LSTM, the
detection accuracy is not much higher.
(a)
(b)
(c)
(d)
Fig. 6. Interference detection performance of three models: a is the detection performance of QAM-FM signal, b is the detection performance of QAM-QPSK signal, c is
the detection performance of QAM-QAM signal, d is the detection performance of
FM-DSSS signal
(a)
(b)
(c)
(d)
Fig. 7. Interference detection performance of BI-LSTM (noiseless prediction) under
different evaluation criteria: a is the detection performance of QAM-FM signal, b is
the detection performance of QAM-QPSK signal, c is the detection performance of
QAM-QAM signal, d is the detection performance of FM-DSSS signal
Figure 7 shows the experimental results of BI-LSTM when the noiseless signal as label. We can see that the detection precision and recall significantly be
improved with the increase of SINR. Almost all the interferences are determined
to be normal when the SINR = 0 db. When the SINR is higher than 6 db, perfect
detection performance is achieved. Among the four scenarios, only the QAM
interference with different symbol rates is the most difficultly to detect. The
DSSS interference in FM modulation is the most easily to detect, which shows
that the signal of digital modulation is hard to predict because of the randomness
of its own symbols.
Q. Wu et al.
4.2 Performance Evaluation
We repeat these experiences in three models above-mentioned and get Fig. 6
which show the F1 score of the performance of three models under different
SINR. F 1 score is calculated as F 1 score =
2P R
P +R (where P represents precision and
R represents recall). Obviously, when using the noiseless signal as label, we get
the best performance, especially at low SINR. In the case of noise label, although
the MAE and MAPE values of Bi-LSTM are smaller than those of LSTM, the
detection accuracy is not much higher.
(a)
(b)
(c)
(d)
Fig. 6. Interference detection performance of three models: a is the detection performance of QAM-FM signal, b is the detection performance of QAM-QPSK signal, c is
the detection performance of QAM-QAM signal, d is the detection performance of
FM-DSSS signal
(a)
(b)
(c)
(d)
Fig. 7. Interference detection performance of BI-LSTM (noiseless prediction) under
different evaluation criteria: a is the detection performance of QAM-FM signal, b is
the detection performance of QAM-QPSK signal, c is the detection performance of
QAM-QAM signal, d is the detection performance of FM-DSSS signal
Figure 7 shows the experimental results of BI-LSTM when the noiseless signal as label. We can see that the detection precision and recall significantly be
improved with the increase of SINR. Almost all the interferences are determined
to be normal when the SINR = 0 db. When the SINR is higher than 6 db, perfect
detection performance is achieved. Among the four scenarios, only the QAM
interference with different symbol rates is the most difficultly to detect. The
DSSS interference in FM modulation is the most easily to detect, which shows
that the signal of digital modulation is hard to predict because of the randomness
of its own symbols.