7 Intelligent and Connected Cyber-Physical Systems: A Perspective. . .
383
also to understand under which sets of conditions the function does not meet its
expectations. This information can be used to design redundancy concepts and runtime measures. For example:
• Run-time plausibility checks: Plausibility checks on the outputs of the neural
network could involve tracking results over time (e.g., objects detected in one
frame should appear in contiguous frames, until out of view) or by comparing
against alternative sensor inputs (e.g., radar or LIDAR reflections). Such plausibility checks may mitigate against inaccuracies that occur spontaneously for
individual frames.
• Run-time monitoring of assumptions: If certain assumptions regarding the operational distribution are determined to be critical, then they could also be monitored
during run-time. Discrepancies between the distribution of objects detected at
run-time and the assumptions could indicate either errors in the trained function
or that the system is operating within a context for which it was not adequately
trained. If such a situation is detected, appropriate actions for mitigating the effect
of the discrepancy can be initiated.
7.3.1.5 Summary
The strategy for arguing the safety of an automated driving system that makes
use of machine learning in its perception functions is summarized in Fig. 7.11. A
systematic analysis of the operating domain is required to identify classes of relevant
scenarios and environmental characteristics that could impact the performance of
the function. This includes explicitly stating assumptions made when selecting
the training data (e.g., typical size and appearance of pedestrians). Based on a
system-level understanding of the functional and performance requirements, a set
of specific requirements must be derived and allocated to the machine learning
function based on an understanding of the inherent performance potential given
Fig. 7.11 Summary of assurance approach for machine learning in automated driving
383
also to understand under which sets of conditions the function does not meet its
expectations. This information can be used to design redundancy concepts and runtime measures. For example:
• Run-time plausibility checks: Plausibility checks on the outputs of the neural
network could involve tracking results over time (e.g., objects detected in one
frame should appear in contiguous frames, until out of view) or by comparing
against alternative sensor inputs (e.g., radar or LIDAR reflections). Such plausibility checks may mitigate against inaccuracies that occur spontaneously for
individual frames.
• Run-time monitoring of assumptions: If certain assumptions regarding the operational distribution are determined to be critical, then they could also be monitored
during run-time. Discrepancies between the distribution of objects detected at
run-time and the assumptions could indicate either errors in the trained function
or that the system is operating within a context for which it was not adequately
trained. If such a situation is detected, appropriate actions for mitigating the effect
of the discrepancy can be initiated.
7.3.1.5 Summary
The strategy for arguing the safety of an automated driving system that makes
use of machine learning in its perception functions is summarized in Fig. 7.11. A
systematic analysis of the operating domain is required to identify classes of relevant
scenarios and environmental characteristics that could impact the performance of
the function. This includes explicitly stating assumptions made when selecting
the training data (e.g., typical size and appearance of pedestrians). Based on a
system-level understanding of the functional and performance requirements, a set
of specific requirements must be derived and allocated to the machine learning
function based on an understanding of the inherent performance potential given
Fig. 7.11 Summary of assurance approach for machine learning in automated driving