204
Sensemaking in Safety Critical and Complex Situations
Technology in autonomous systems and their interpretation ( such as through AI)
are not reliable at p resent – thus, there is a need to address poor reliability trough
improving man/ technology/ organisation aspects. The reliability of drones is lower
than for manned planes, and we have seen how sensors and technical equipment are
causing safety issues in several projects. The systems must improve for an industrial
setting and for safety-critical operations, i.e. become highly reliable and resilient
to bad data and have automatic s elf-checking behaviour and avoiding single-point
failures by checking across multiple inputs. Thus, there is a need to get support
from other AVs with sensors, need for developing infrastructure ( such as roads and
seaways with sensors), in addition to establishment of control centres for road traffic
and maritime traffic that must be responsible for supporting sensemaking among
the actors ( i.e. automated and not automated systems). Technical barriers must be
in place to a larger extent on autonomous systems to avoid and reduce the outcome
of failures and component interaction accidents, which are more common as the
complexity increases.
Automation transparency is important for both sharing the situation awareness
and communicating the intentions towards others and for the operator in an RCC
to understand the behaviour of the automation. In complex systems, a wide range of
alarm issues related to diagnostics, management and assessments of multiple input
data will be challenging. Hence, alarms must be unambiguous and displayed with
a clear message. This requires good human factor engineering practice, such as an
alarm philosophy and relevant standards.
Organisation: Experience from the projects and pilots demonstrate a need to
see the technological solution in a larger sociotechnical context. Autonomous
transportation systems are a system of systems. We have seen that legislation is
are needed to gather data and establish the operational context. There is a need for
substantial investments in infrastructure: organisational interfaces are lacking and
organisational/ structural issues from the operator/ company/ area/ society are often
considered the last thing to get in place. Looking at the operational context, we have
seen a need to limit the operational design domain and use operational envelopes, or
safety envelopes to define situations, responsibilities and system characteristics during all conditions ( especially in safety-critical conditions with sensor/ data failures).
Regulations and guidelines have slowly been established to support autonomous
transportation systems. However, few of them require systematic reporting of accidents and incidents. Experience from accidents with AVs has given valuable insight,
and hence all domains should prioritise and require reporting and systematic data
collection of failures, hazards and unforeseen events. Not requiring reporting and
sharing of safety-critical systems is a risk in itself.
SENSEMAKING TO SUPPORT MEANINGFUL HUMAN CONTROL
Focus on the design of operational envelopes to reduce complexity and analysing
the needs for cues and information to support sensemaking and meaningful human
control, when needed, is a key issue. Defining operational envelopes answers the
question of which functions and roles automation/ autonomy should have, versus
Sensemaking in Safety Critical and Complex Situations
Technology in autonomous systems and their interpretation ( such as through AI)
are not reliable at p resent – thus, there is a need to address poor reliability trough
improving man/ technology/ organisation aspects. The reliability of drones is lower
than for manned planes, and we have seen how sensors and technical equipment are
causing safety issues in several projects. The systems must improve for an industrial
setting and for safety-critical operations, i.e. become highly reliable and resilient
to bad data and have automatic s elf-checking behaviour and avoiding single-point
failures by checking across multiple inputs. Thus, there is a need to get support
from other AVs with sensors, need for developing infrastructure ( such as roads and
seaways with sensors), in addition to establishment of control centres for road traffic
and maritime traffic that must be responsible for supporting sensemaking among
the actors ( i.e. automated and not automated systems). Technical barriers must be
in place to a larger extent on autonomous systems to avoid and reduce the outcome
of failures and component interaction accidents, which are more common as the
complexity increases.
Automation transparency is important for both sharing the situation awareness
and communicating the intentions towards others and for the operator in an RCC
to understand the behaviour of the automation. In complex systems, a wide range of
alarm issues related to diagnostics, management and assessments of multiple input
data will be challenging. Hence, alarms must be unambiguous and displayed with
a clear message. This requires good human factor engineering practice, such as an
alarm philosophy and relevant standards.
Organisation: Experience from the projects and pilots demonstrate a need to
see the technological solution in a larger sociotechnical context. Autonomous
transportation systems are a system of systems. We have seen that legislation is
are needed to gather data and establish the operational context. There is a need for
substantial investments in infrastructure: organisational interfaces are lacking and
organisational/ structural issues from the operator/ company/ area/ society are often
considered the last thing to get in place. Looking at the operational context, we have
seen a need to limit the operational design domain and use operational envelopes, or
safety envelopes to define situations, responsibilities and system characteristics during all conditions ( especially in safety-critical conditions with sensor/ data failures).
Regulations and guidelines have slowly been established to support autonomous
transportation systems. However, few of them require systematic reporting of accidents and incidents. Experience from accidents with AVs has given valuable insight,
and hence all domains should prioritise and require reporting and systematic data
collection of failures, hazards and unforeseen events. Not requiring reporting and
sharing of safety-critical systems is a risk in itself.
SENSEMAKING TO SUPPORT MEANINGFUL HUMAN CONTROL
Focus on the design of operational envelopes to reduce complexity and analysing
the needs for cues and information to support sensemaking and meaningful human
control, when needed, is a key issue. Defining operational envelopes answers the
question of which functions and roles automation/ autonomy should have, versus