203
Improving Safety
( sensors), programming ( rule-based and not artificial intelligence, AI), prolonged
attention ( humans in the loop), HMI ( Autopilot-engagement rules) and misuse. The
list may become longer as more safety data are gathered and more i n-depth information on accident causality of automated vehicles is established, e.g. overreliance and
expectation mismatch.
Based on the experiences, there is a need to establish regulations that ensure systematic incident reporting, develop systems based on learning from incidents and
invest in infrastructure to support automation, i.e. help the automation by focussing
on an operational envelope that uses more data from infrastructure. The transport
systems are automated but not autonomous. Autonomous systems are immature at
present and must be further developed.
A SUMMARY OF MTO SAFETY ISSUES
Based on the performed reviews, the suggested key measures are listed below.
Humans: As seen from all experiences, the uncertain and complex environment
for autonomous systems must ensure the need for human intervention. Autonomous
transportation systems will to a varying degree need human control if failures occur or
under certain operational conditions. With today’s UTOs and AGVs, an operator is still
needed when there is a disruption and sensors fail to detect and recognise an obstacle
or determine the next actions. However, in testing and developing autonomous transportation systems with drones, AVs and vessels, we see examples of projects where
the human operator is not considered from the beginning. The industries’ motivation
seems to be to try to automate as much as possible and assume that humans will and
can monitor it. Hence, HAI and how to keep the humans in the loop is often considered
a challenge to be solved late in the project after knowing the limitations of the technology and by considering the humans as the adapting back-up. Most of the projects
lack early incorporation of human factors in analysis, design, testing and certification
process. Thus, there are costly challenges that should have been addressed earlier by
starting with technology, human limitations and possibilities, and organisational and
infrastructure needs. A key issue is to define the design conditions the system should
operate under by defining the operational envelope and critical scenarios ( such as sensor failures). Then specify how critical scenarios can be mitigated by infrastructure
support i.e. surrounding systems such as other autonomous systems nearby ( cars) or
control infrastructure. If human intervention is needed to handle the scenarios, sensemaking must be supported within the existing limitation of human abilities.
As aviation is the industry with the most experience with safe automated systems,
the list from Endsley ( 2019) with design principles for improving people’s ability to
successfully oversee and interact with automated systems should be a very useful element, allowing for manual overrides and sufficient training to users on automation to
ensure adequate understanding and appropriate levels of trust.
Technology: To date, developing autonomous or remotely controlled transportation systems ( especially for AVs and MASS) appears to primarily be about a technology push rather than considering and providing sociotechnical solutions including
redesign of work, capturing knowledge and addressing human factors as we and
others have seen ( Lutzhoft et al., 2019).
Improving Safety
( sensors), programming ( rule-based and not artificial intelligence, AI), prolonged
attention ( humans in the loop), HMI ( Autopilot-engagement rules) and misuse. The
list may become longer as more safety data are gathered and more i n-depth information on accident causality of automated vehicles is established, e.g. overreliance and
expectation mismatch.
Based on the experiences, there is a need to establish regulations that ensure systematic incident reporting, develop systems based on learning from incidents and
invest in infrastructure to support automation, i.e. help the automation by focussing
on an operational envelope that uses more data from infrastructure. The transport
systems are automated but not autonomous. Autonomous systems are immature at
present and must be further developed.
A SUMMARY OF MTO SAFETY ISSUES
Based on the performed reviews, the suggested key measures are listed below.
Humans: As seen from all experiences, the uncertain and complex environment
for autonomous systems must ensure the need for human intervention. Autonomous
transportation systems will to a varying degree need human control if failures occur or
under certain operational conditions. With today’s UTOs and AGVs, an operator is still
needed when there is a disruption and sensors fail to detect and recognise an obstacle
or determine the next actions. However, in testing and developing autonomous transportation systems with drones, AVs and vessels, we see examples of projects where
the human operator is not considered from the beginning. The industries’ motivation
seems to be to try to automate as much as possible and assume that humans will and
can monitor it. Hence, HAI and how to keep the humans in the loop is often considered
a challenge to be solved late in the project after knowing the limitations of the technology and by considering the humans as the adapting back-up. Most of the projects
lack early incorporation of human factors in analysis, design, testing and certification
process. Thus, there are costly challenges that should have been addressed earlier by
starting with technology, human limitations and possibilities, and organisational and
infrastructure needs. A key issue is to define the design conditions the system should
operate under by defining the operational envelope and critical scenarios ( such as sensor failures). Then specify how critical scenarios can be mitigated by infrastructure
support i.e. surrounding systems such as other autonomous systems nearby ( cars) or
control infrastructure. If human intervention is needed to handle the scenarios, sensemaking must be supported within the existing limitation of human abilities.
As aviation is the industry with the most experience with safe automated systems,
the list from Endsley ( 2019) with design principles for improving people’s ability to
successfully oversee and interact with automated systems should be a very useful element, allowing for manual overrides and sufficient training to users on automation to
ensure adequate understanding and appropriate levels of trust.
Technology: To date, developing autonomous or remotely controlled transportation systems ( especially for AVs and MASS) appears to primarily be about a technology push rather than considering and providing sociotechnical solutions including
redesign of work, capturing knowledge and addressing human factors as we and
others have seen ( Lutzhoft et al., 2019).