10 Challenges in Certifying Small-Scale (IoT) Hardware Random Number Generators
179
Table 10.7 ENT results
Bytes
Bits
Samples
χ 2
Serial corr.
χ 2
Serial corr.
Device
#
passed
passed
passed
passed
Quantis 16M
100
10
99
0
100
Post 16M
100
100
96
100
96
Quantis 4M
100
0
99
0
49
Post 4M
100
100
99
100
100
Quantis USB
100
0
92
0
81
Post USB
100
100
94
100
100
Comscire PQ32MU
100
100
99
100
100
samples. The bias appears to drift between samples, with the only constant being a
tendency to express a 10 −6 bias above the normal for byte values 0–5. Even this is
not a representative trend, with only 38% of samples showing this particular trait.
Figure 10.3 shows the χ 2 statistics for raw Quantis samples from all the 16M and
4M devices. The results for the Quantis USB are omitted, as the USB is effectively
a 4M in different packaging and provides similar results.
The 16M (a) fails the χ 2 test for 90 of its samples. The mean statistic for the 16M
is approximately 350. This is above the acceptable maximum threshold for this test.
The 4M is significantly worse, with a mean statistic of 506. Unlike the 16M, the
4M shows no passes at all (the USB reports similar results). In fact, the minimum
statistic for the 4M was 407. This is significantly above the maximum threshold for
the χ 2 test.
The experiments conducted over these QRNGs show that established tests do
not always identify issues that more recent (or just less well-known) tests highlight.
The TestU01 battery reinforces the results of the Ent test, by providing a wider
variety of more sophisticated tests that prove that there are issues beyond simple
deviation from the normal distribution of values at the byte and bit level. As TestU01
is designed to provide the tools to test TRNG, this battery would ideally be made
a mandatory recommendation for TRNG and QRNG testing. Dieharder and NIST
SP800-22 will remain in use, as they are effective at identifying egregious issues
with RNG output, but the extension of the minimum recommended number of tests
is very much needed at this time. Post-processed and raw data should be tested and
the results clearly marked to show users how the improper configuration of software
post-processing can be identified and resolved. One should also consider that if
IoT QRNGs are sought-after, how does one implement a post-processing algorithm
(which are known for their high memory requirements) in such a small package?
Resource limitations may prevent effective post-processing of QRNG output, the
consequences of which are made clear in the preceding work.
179
Table 10.7 ENT results
Bytes
Bits
Samples
χ 2
Serial corr.
χ 2
Serial corr.
Device
#
passed
passed
passed
passed
Quantis 16M
100
10
99
0
100
Post 16M
100
100
96
100
96
Quantis 4M
100
0
99
0
49
Post 4M
100
100
99
100
100
Quantis USB
100
0
92
0
81
Post USB
100
100
94
100
100
Comscire PQ32MU
100
100
99
100
100
samples. The bias appears to drift between samples, with the only constant being a
tendency to express a 10 −6 bias above the normal for byte values 0–5. Even this is
not a representative trend, with only 38% of samples showing this particular trait.
Figure 10.3 shows the χ 2 statistics for raw Quantis samples from all the 16M and
4M devices. The results for the Quantis USB are omitted, as the USB is effectively
a 4M in different packaging and provides similar results.
The 16M (a) fails the χ 2 test for 90 of its samples. The mean statistic for the 16M
is approximately 350. This is above the acceptable maximum threshold for this test.
The 4M is significantly worse, with a mean statistic of 506. Unlike the 16M, the
4M shows no passes at all (the USB reports similar results). In fact, the minimum
statistic for the 4M was 407. This is significantly above the maximum threshold for
the χ 2 test.
The experiments conducted over these QRNGs show that established tests do
not always identify issues that more recent (or just less well-known) tests highlight.
The TestU01 battery reinforces the results of the Ent test, by providing a wider
variety of more sophisticated tests that prove that there are issues beyond simple
deviation from the normal distribution of values at the byte and bit level. As TestU01
is designed to provide the tools to test TRNG, this battery would ideally be made
a mandatory recommendation for TRNG and QRNG testing. Dieharder and NIST
SP800-22 will remain in use, as they are effective at identifying egregious issues
with RNG output, but the extension of the minimum recommended number of tests
is very much needed at this time. Post-processed and raw data should be tested and
the results clearly marked to show users how the improper configuration of software
post-processing can be identified and resolved. One should also consider that if
IoT QRNGs are sought-after, how does one implement a post-processing algorithm
(which are known for their high memory requirements) in such a small package?
Resource limitations may prevent effective post-processing of QRNG output, the
consequences of which are made clear in the preceding work.