10 Challenges in Certifying Small-Scale (IoT) Hardware Random Number Generators
177
at sample sizes smaller than 7.5 KB [288, 289]. This emphasizes the point made
in Sect. 3 regarding data collection. The amount of data collected needs to meet a
minimum size to reliably identify issues in the RNGs being tested. This minimum
threshold is test-specific, requiring that the highest minimum sample size is
identified by analysts prior to conducting an evaluation of an RNG. Furthermore,
these experiments showed that even well-respected, proven statistical test batteries
such as Dieharder and NIST SP800-22 were unable to identify the issue with the
DESFire EV1. It is clear that it is possible to design a TRNG to pass these tests,
but is it wise to rely on tests that can be designed for? Does designing to meet the
finite and narrow requirements of Dieharder and NIST SP800-22 actually provide
any guarantees of randomness? We argue that it does not.
10.4.2 Identifying Issues with Quantum Random Number
Generators
The EV1 experiments provided an introduction to issues in using well-established
statistical tests to identify non-randomness in TRNG. QRNGs are currently too
large for RFID card or IoT device implementations, but miniaturization of quantum
entropy sources is proceeding quickly, and proposals for IoT-scale QRNG have
already been published. However, there are several open problems with the current
generation of QRNGs and their evaluation.
Even when sample collection is not a problem, there can be issues. IDQ’s Quantis
range of QRNGs is based on an optical quantum source of entropy (a beam splitter).
Comscire produces a rival product, the PQ32MU, which uses quantum shot-noise as
its entropy source. Both companies provide multiple models of QRNG with varying
speeds, all with appropriate statistical test results associated with their devices.
As previously discussed, IDQ provides a relatively robust test report, though it is
limited to Diehard and NIST SP800-22 tests. Comscire uses few and small samples,
with a smaller number of tests, limiting the rigor of its test process significantly.
None of the devices tested as a part of this work were validated using the AIS31 methodology, nor were they certified (as there are no official certifications for
standalone RNGs).
Data collection is not an issue from these devices, the Quantis devices provide
data at a rate of 4 or 16 Mb/s, whiles the PQ32MU has an output speed of 32 Mb/s.
As a result, collecting large amounts of data is trivial. A key difference in these two
brands is that the Quantis generators do not implement on-board post-processing
to remove bias, whilst the PQ32MU is an all-in-one product with post-processing
performed on-device.
Table 10.6 shows the results for the EV1, Quantis generators and PQ32MU. Both
raw and post-processed Quantis output is shown. EV1 data is tested for 64 MB
samples over 100 cards. Quantis and Comscire QRNGs are tested over 100 2.1 GB
177
at sample sizes smaller than 7.5 KB [288, 289]. This emphasizes the point made
in Sect. 3 regarding data collection. The amount of data collected needs to meet a
minimum size to reliably identify issues in the RNGs being tested. This minimum
threshold is test-specific, requiring that the highest minimum sample size is
identified by analysts prior to conducting an evaluation of an RNG. Furthermore,
these experiments showed that even well-respected, proven statistical test batteries
such as Dieharder and NIST SP800-22 were unable to identify the issue with the
DESFire EV1. It is clear that it is possible to design a TRNG to pass these tests,
but is it wise to rely on tests that can be designed for? Does designing to meet the
finite and narrow requirements of Dieharder and NIST SP800-22 actually provide
any guarantees of randomness? We argue that it does not.
10.4.2 Identifying Issues with Quantum Random Number
Generators
The EV1 experiments provided an introduction to issues in using well-established
statistical tests to identify non-randomness in TRNG. QRNGs are currently too
large for RFID card or IoT device implementations, but miniaturization of quantum
entropy sources is proceeding quickly, and proposals for IoT-scale QRNG have
already been published. However, there are several open problems with the current
generation of QRNGs and their evaluation.
Even when sample collection is not a problem, there can be issues. IDQ’s Quantis
range of QRNGs is based on an optical quantum source of entropy (a beam splitter).
Comscire produces a rival product, the PQ32MU, which uses quantum shot-noise as
its entropy source. Both companies provide multiple models of QRNG with varying
speeds, all with appropriate statistical test results associated with their devices.
As previously discussed, IDQ provides a relatively robust test report, though it is
limited to Diehard and NIST SP800-22 tests. Comscire uses few and small samples,
with a smaller number of tests, limiting the rigor of its test process significantly.
None of the devices tested as a part of this work were validated using the AIS31 methodology, nor were they certified (as there are no official certifications for
standalone RNGs).
Data collection is not an issue from these devices, the Quantis devices provide
data at a rate of 4 or 16 Mb/s, whiles the PQ32MU has an output speed of 32 Mb/s.
As a result, collecting large amounts of data is trivial. A key difference in these two
brands is that the Quantis generators do not implement on-board post-processing
to remove bias, whilst the PQ32MU is an all-in-one product with post-processing
performed on-device.
Table 10.6 shows the results for the EV1, Quantis generators and PQ32MU. Both
raw and post-processed Quantis output is shown. EV1 data is tested for 64 MB
samples over 100 cards. Quantis and Comscire QRNGs are tested over 100 2.1 GB