10 Challenges in Certifying Small-Scale (IoT) Hardware Random Number Generators
169
number of tests. Alphabits and Rabbit operate over bits, whilst the Crush batteries
operate over floating point numbers between 0 and 1. Alphabits, Rabbit, and Small
Crush complete in minutes over samples of 2 GB in size. Big Crush requires a large
amount of data (or a constant stream of input from the target device) and can take
hours to complete. McCullough et al. identify some potential issues with this toolset. Some tests are only able to read 32 bits and are more sensitive to errors in
their most significant bits than their least significant bits [403]. To resolve this, they
suggest that tests are performed over the sequences forwards and backwards. The
issue here is that a test on live data cannot be performed in this manner. This limits
many tests and prevents them from being used as live tests.
Another class of tests exists; continuous tests. These tests are designed to identify
whether there have been hardware failures that lead to corruption or cessation
of the entropy stream. FIPS 140-1/2 are designed with hardware in mind [121].
Both tests suites can be implemented in the circuitry of an RNG, providing a
constant series of results regarding the health and functionality of the device. A core
requirement of any continuous test is that no RNG should output two identical N bit
blocks in succession. If this condition is not met, the device should cease function
immediately and alert the user that it is not performing as expected. However,
this does not detect more subtle flaws. The astute reader may also have deduced
that requiring that no two N bit blocks be identical actually results in reduced
entropy. This has an impact on the legitimacy of such tests when considering that
the definition of an ideal RNG is one that is completely unpredictable. These tests
are likely to be implemented alongside IoT TRNG implementations due to their
efficient implementation in hardware, carrying the previous concerns to millions of
potential devices.
The usage of NIST, Dieharder, TestU01, and other statistical test batteries can
vary between institutions. NIST SP800-22 outlines minimum sample sizes and
Dieharder implies these by rewinding samples if insufficient data is provided.
However, during self-certification, some companies have been found to test small
samples, below the suggested guidelines. This can cast doubt over the validity of
their findings.
10.3 Challenges in Data Collection
For standalone RNG, data collection may be simple. However, there are no official
certifications for standalone RNG. FIPS and CC both certify whole security systems,
not individual elements, so even though RNG testing is key to this process, a
standalone RNG that passes these tests still cannot be certified. Regardless, RNG
testing as a part of whole system certification is a critical consideration. Data
collection from certain integrated RNGs may not be trivial. As IoT devices represent
a whole-system security implementation, they may be certified; RNG evaluation
forms a critical part of any such evaluation process.
169
number of tests. Alphabits and Rabbit operate over bits, whilst the Crush batteries
operate over floating point numbers between 0 and 1. Alphabits, Rabbit, and Small
Crush complete in minutes over samples of 2 GB in size. Big Crush requires a large
amount of data (or a constant stream of input from the target device) and can take
hours to complete. McCullough et al. identify some potential issues with this toolset. Some tests are only able to read 32 bits and are more sensitive to errors in
their most significant bits than their least significant bits [403]. To resolve this, they
suggest that tests are performed over the sequences forwards and backwards. The
issue here is that a test on live data cannot be performed in this manner. This limits
many tests and prevents them from being used as live tests.
Another class of tests exists; continuous tests. These tests are designed to identify
whether there have been hardware failures that lead to corruption or cessation
of the entropy stream. FIPS 140-1/2 are designed with hardware in mind [121].
Both tests suites can be implemented in the circuitry of an RNG, providing a
constant series of results regarding the health and functionality of the device. A core
requirement of any continuous test is that no RNG should output two identical N bit
blocks in succession. If this condition is not met, the device should cease function
immediately and alert the user that it is not performing as expected. However,
this does not detect more subtle flaws. The astute reader may also have deduced
that requiring that no two N bit blocks be identical actually results in reduced
entropy. This has an impact on the legitimacy of such tests when considering that
the definition of an ideal RNG is one that is completely unpredictable. These tests
are likely to be implemented alongside IoT TRNG implementations due to their
efficient implementation in hardware, carrying the previous concerns to millions of
potential devices.
The usage of NIST, Dieharder, TestU01, and other statistical test batteries can
vary between institutions. NIST SP800-22 outlines minimum sample sizes and
Dieharder implies these by rewinding samples if insufficient data is provided.
However, during self-certification, some companies have been found to test small
samples, below the suggested guidelines. This can cast doubt over the validity of
their findings.
10.3 Challenges in Data Collection
For standalone RNG, data collection may be simple. However, there are no official
certifications for standalone RNG. FIPS and CC both certify whole security systems,
not individual elements, so even though RNG testing is key to this process, a
standalone RNG that passes these tests still cannot be certified. Regardless, RNG
testing as a part of whole system certification is a critical consideration. Data
collection from certain integrated RNGs may not be trivial. As IoT devices represent
a whole-system security implementation, they may be certified; RNG evaluation
forms a critical part of any such evaluation process.