7 Intelligent and Connected Cyber-Physical Systems: A Perspective. . .
377
human abilities. Neither of the above-mentioned standards address the application
of machine learning techniques to automated driving tasks. As a result, assurance
methods must be developed and the ability of the system to meet its safety goals
must be systematically argued based on “first principles” where adherence to a
standard is only one part of the overall argument. An assurance case [12] provides a
convincing and valid argument that a set of claims regarding the safety of a system
is justified for a given function based on a set of assumptions over its operational
context.
The rest of this case study is structured as follows. First, safety requirements
allocated to machine learning functions are described from a systems engineering
perspective and an example function, camera-based pedestrian recognition, is
introduced. Next, the potential causes of functional insufficiencies in machine
learning functions are discussed, for which mitigation measures will form a key
component of the safety assurance argument. A number of sources of evidence
of the performance of the machine learning functions are then described. Finally,
we reflect on the current state of the art in this area and the overall potential for
providing a convincing argument for using machine learning technique in safetycritical applications.
7.3.1.2 Safety Requirements on the Machine Learning Function
The challenges involved in providing a convincing system-level assurance case
will depend on the functional scope of the machine learning application as well
as whether it is trained and validated during development, or whether it continues
to learn in the field. It is expected that, in practice, the initial applications of
machine learning in series development of highly automated driving will be based
on pre-trained functions, implementing well-specified detection tasks which can be
supported by plausibility checks based on alternative channels within the system
context. One such example application, which shall be referenced in the rest of this
case study, is the application of Convolutional Neural Networks (CNNs) [11] to
detect objects such as pedestrians based on camera images as part of a collision
avoidance system for self-driving vehicles. CNNs are a class of feedforward neural
networks (NN) that consist of a large number of connected neurons – computational
units that calculate a weighted sum of their inputs and apply a nonlinear activation
function on this sum. The weights are determined by minimizing a loss function of
the network over a given set of training data (labelled images) and back-propagating
the respective error terms through the network. In this manner, CNNs allow a
classification annotated with a confidence level for each class and a localization
of an object within a given image (e.g., frames of a video).
Performance requirements must be defined and allocated to the machine learning
function in order to ensure that, at a system level, the safety goals are met.
The derivation of performance (Safety) requirements within the system context is
one of the key contributions to ensuring overall system safety and requires deep
domain and system knowledge. Deriving a suitable set of requirements for open
377
human abilities. Neither of the above-mentioned standards address the application
of machine learning techniques to automated driving tasks. As a result, assurance
methods must be developed and the ability of the system to meet its safety goals
must be systematically argued based on “first principles” where adherence to a
standard is only one part of the overall argument. An assurance case [12] provides a
convincing and valid argument that a set of claims regarding the safety of a system
is justified for a given function based on a set of assumptions over its operational
context.
The rest of this case study is structured as follows. First, safety requirements
allocated to machine learning functions are described from a systems engineering
perspective and an example function, camera-based pedestrian recognition, is
introduced. Next, the potential causes of functional insufficiencies in machine
learning functions are discussed, for which mitigation measures will form a key
component of the safety assurance argument. A number of sources of evidence
of the performance of the machine learning functions are then described. Finally,
we reflect on the current state of the art in this area and the overall potential for
providing a convincing argument for using machine learning technique in safetycritical applications.
7.3.1.2 Safety Requirements on the Machine Learning Function
The challenges involved in providing a convincing system-level assurance case
will depend on the functional scope of the machine learning application as well
as whether it is trained and validated during development, or whether it continues
to learn in the field. It is expected that, in practice, the initial applications of
machine learning in series development of highly automated driving will be based
on pre-trained functions, implementing well-specified detection tasks which can be
supported by plausibility checks based on alternative channels within the system
context. One such example application, which shall be referenced in the rest of this
case study, is the application of Convolutional Neural Networks (CNNs) [11] to
detect objects such as pedestrians based on camera images as part of a collision
avoidance system for self-driving vehicles. CNNs are a class of feedforward neural
networks (NN) that consist of a large number of connected neurons – computational
units that calculate a weighted sum of their inputs and apply a nonlinear activation
function on this sum. The weights are determined by minimizing a loss function of
the network over a given set of training data (labelled images) and back-propagating
the respective error terms through the network. In this manner, CNNs allow a
classification annotated with a confidence level for each class and a localization
of an object within a given image (e.g., frames of a video).
Performance requirements must be defined and allocated to the machine learning
function in order to ensure that, at a system level, the safety goals are met.
The derivation of performance (Safety) requirements within the system context is
one of the key contributions to ensuring overall system safety and requires deep
domain and system knowledge. Deriving a suitable set of requirements for open