324
D. Zhu et al.
4 Comprehensive Evaluation
We define the following test indexes:
1. Voice activity detection correct ratio (R vad ), the ratio of the number of correct
speech samples detected by the system to all samples number:
R vad =
N
i=1 C vad
N
2. Speech recognition correct ratio (R asr ), word correct rate:
R asr =
N
i=1 C asr
N
3. Instruction analysis correct ratio (R iacr ), the ratio of the number of the correct
instruction analyzed by the system to all samples number:
R iacr =
N
i=1 C iacr
N
In order to test the system comprehensive performance, we build the simulation test corpus. We firstly build speech samples set that includes samples
collected from 150 people’s speech samples, THCH30 (A Free Chinese Speech
Corpus Released by Tsinghua University), and Baidu Voice Synthesis Set, and
then we randomly extract samples from the set, add random intervals, and combine they with additive noise. Figure 3 is the table of speech recognition correct (Rasr) and instruction analysis correct ratio (Riacr) results in the test.
Figure 4 shows the character error ratio (CER) and word error ratio (WER)
during training. By comparing these results, it is clear that the 4th plan has the
best comprehensive performance.
Fig. 3. Result table
D. Zhu et al.
4 Comprehensive Evaluation
We define the following test indexes:
1. Voice activity detection correct ratio (R vad ), the ratio of the number of correct
speech samples detected by the system to all samples number:
R vad =
N
i=1 C vad
N
2. Speech recognition correct ratio (R asr ), word correct rate:
R asr =
N
i=1 C asr
N
3. Instruction analysis correct ratio (R iacr ), the ratio of the number of the correct
instruction analyzed by the system to all samples number:
R iacr =
N
i=1 C iacr
N
In order to test the system comprehensive performance, we build the simulation test corpus. We firstly build speech samples set that includes samples
collected from 150 people’s speech samples, THCH30 (A Free Chinese Speech
Corpus Released by Tsinghua University), and Baidu Voice Synthesis Set, and
then we randomly extract samples from the set, add random intervals, and combine they with additive noise. Figure 3 is the table of speech recognition correct (Rasr) and instruction analysis correct ratio (Riacr) results in the test.
Figure 4 shows the character error ratio (CER) and word error ratio (WER)
during training. By comparing these results, it is clear that the 4th plan has the
best comprehensive performance.
Fig. 3. Result table