10 Challenges in Certifying Small-Scale (IoT) Hardware Random Number Generators
167
10.2 Certification, Standards, and Testing
Many companies employ their own testing teams, to whom the responsibility of
carrying out company mandated quality control falls. ID Quantique (IDQ) and
NXP are two examples, both of whom perform varying degrees of testing on their
products. In the case of products implementing cryptography, RNG testing is vital
for the validation of the cryptosystem in question. However, in-house testing is
insufficient for certification, with the exception of self-certification (as performed
by IDQ). Testing must be performed by a third-party to ensure impartiality.
NIST is one example of a standards and testing institution. This US institute
concerns itself with the advancement of measurement science, standards and
technology. This body does not conduct testing or reward certificates itself but is
responsible for the publication and impartial development of statistical test suites
for randomness tests. Special Publications (SP) are created to circulate accepted
developments in the field of RNG testing and formal verification of RNG. Of
particular note are SP800-90B [449] and SP800-22 [448]. SP800-90B details
specific tests for the entropy source and final outputs of PRNG and TRNG. SP80022 details an extensive test battery suitable for use over PRNG and TRNG (including
QRNG by association with TRNG).
Common Criteria (CC) is an international standard (ISO/IEC 15408). Unlike
the NIST SP documents discussed previously, CC is a broad framework for the
verification of computer security systems [407]. Functionality, construction, and
assurance requirements are the core tenets of the CC. It is important to emphasize
that this is a whole-system-security verification: RNG testing is only part of a larger
verification process. However, it can be argued that RNG validation is a keystone
for the certification of a computer-based security system. If the RNG is incapable of
providing the appropriate output, then it is unlikely that the security system will be
robust to the degree demanded by the CC.
To differentiate between different applications and their security requirements,
the CC has developed the Evaluation Assurance Level (EAL) scheme. These
numbered levels, from 1 to 7, reflect an increasing security requirement. At level 1,
testing is cursory and reports provided by manufacturers are acceptable. As higher
certifications are sought, more third party and design-stage tests by third parties
are required. At levels 5+, spot checks of manufacturing plants and implementation
of security critical systems are performed. NXP produces two CC EAL certified
devices: the DESFire EV1 (EAL4+), and the DESFire EV2 (EAL5+).
The test methodology employed by the CC when testing RNGs is outlined
in AIS-31 [327]. AIS-31 outlines the test methodology for entropy sources in
computer-based security systems [327]. AIS-20 is referred to as the source of
information for recommended tests and parameters for TRNG evaluation. Both
documents have a TRNG focus, as they are aimed at the evaluation of the formal
verification of entropy sources, not the PRNG algorithms that they may seed. As a
result, hardware RNGs are the focus of these documents.
167
10.2 Certification, Standards, and Testing
Many companies employ their own testing teams, to whom the responsibility of
carrying out company mandated quality control falls. ID Quantique (IDQ) and
NXP are two examples, both of whom perform varying degrees of testing on their
products. In the case of products implementing cryptography, RNG testing is vital
for the validation of the cryptosystem in question. However, in-house testing is
insufficient for certification, with the exception of self-certification (as performed
by IDQ). Testing must be performed by a third-party to ensure impartiality.
NIST is one example of a standards and testing institution. This US institute
concerns itself with the advancement of measurement science, standards and
technology. This body does not conduct testing or reward certificates itself but is
responsible for the publication and impartial development of statistical test suites
for randomness tests. Special Publications (SP) are created to circulate accepted
developments in the field of RNG testing and formal verification of RNG. Of
particular note are SP800-90B [449] and SP800-22 [448]. SP800-90B details
specific tests for the entropy source and final outputs of PRNG and TRNG. SP80022 details an extensive test battery suitable for use over PRNG and TRNG (including
QRNG by association with TRNG).
Common Criteria (CC) is an international standard (ISO/IEC 15408). Unlike
the NIST SP documents discussed previously, CC is a broad framework for the
verification of computer security systems [407]. Functionality, construction, and
assurance requirements are the core tenets of the CC. It is important to emphasize
that this is a whole-system-security verification: RNG testing is only part of a larger
verification process. However, it can be argued that RNG validation is a keystone
for the certification of a computer-based security system. If the RNG is incapable of
providing the appropriate output, then it is unlikely that the security system will be
robust to the degree demanded by the CC.
To differentiate between different applications and their security requirements,
the CC has developed the Evaluation Assurance Level (EAL) scheme. These
numbered levels, from 1 to 7, reflect an increasing security requirement. At level 1,
testing is cursory and reports provided by manufacturers are acceptable. As higher
certifications are sought, more third party and design-stage tests by third parties
are required. At levels 5+, spot checks of manufacturing plants and implementation
of security critical systems are performed. NXP produces two CC EAL certified
devices: the DESFire EV1 (EAL4+), and the DESFire EV2 (EAL5+).
The test methodology employed by the CC when testing RNGs is outlined
in AIS-31 [327]. AIS-31 outlines the test methodology for entropy sources in
computer-based security systems [327]. AIS-20 is referred to as the source of
information for recommended tests and parameters for TRNG evaluation. Both
documents have a TRNG focus, as they are aimed at the evaluation of the formal
verification of entropy sources, not the PRNG algorithms that they may seed. As a
result, hardware RNGs are the focus of these documents.