technology, i.e., the foundation of human common sense. Third, there is a wide range
of common sense. Learn from online reading to construct answers to natural language
questions about commonsense phenomena and image-based questions. The service will
simulate the general knowledge of American adults in 2018 and will be judged through
the benchmark test of the Allen Institute of Artificial Intelligence (AI2). DARPA
predicts that researchers will use manual construction, information extraction, machine
learning, crowd sourcing and other computational methods to build the knowledge
base.
3.5 Developing Cyber Resilience to Actively Address the Security
Problem of AI System
In October 2016, the USA released two strategic documents related to the development
of artificial intelligence, the National Strategic Plan for Research and Development of
Artificial Intelligence and Preparing for the Future of Artificial Intelligence. In the area
of AI security, the National AI Research and Development Strategic Plan of the USA
emphasizes the new challenges that traditional network security may bring in AI
environment [10]. It is considered that AI systems embedded in key systems must be
durable and secure to cope with large-scale deliberate network attacks. It is believed
that the application of AI system will bring some new and unique threats. At the same
time, the report also emphasizes that AI systems may eventually adopt “cyclic selfimprovement”; that is, a large number of software modifications will be made by the
software itself, rather than by human programmers. In order to ensure the security of
self-modifying systems, additional research is needed to develop self-monitoring
frameworks such as checking the consistency of the system’s behavior through the
original goal of the designer or proven value frameworks that can resist self-modifying.
In the Report “Summer Study on Autonomy” released by the Defense Science
Board of the United States Department of Defense, the answer to the cyber security
problem of intelligent military applications is to enhance the cyber resilience of
intelligent systems [4].
Cyber resilience can effectively deal with the security problems of AI system
because the focus of cyber resilience shifts from preventing cyber attacks to minimizing
the impact of cyber attacks. It does not seek to establish a perfect system to deal with all
kinds of incidents, including anti-attack. Instead, it adopts a series of technical means to
improve the system’s self-active protection ability, so that the key tasks of the system
can still be carried out normally in the “toxic carrier” environment, under attack or even
under partial control, and maintain the normal function of the system, so as to take the
next step. The measures lay the foundation [3].
In January 2013, the US Department of Defense released the report “Resilient
Military System and Advanced Cyber Threat,” which proposed the construction of
resilient military system. In August 2014, the US Air Force issued a “Cyber Resilience”
research guide, which calls for accelerating the research of resilience mechanism and
key technologies, survival and recovery of core mission functions, cyber deception,
cyber agility, resilience and agility of embedded systems [11].
412
G. Zhou
of common sense. Learn from online reading to construct answers to natural language
questions about commonsense phenomena and image-based questions. The service will
simulate the general knowledge of American adults in 2018 and will be judged through
the benchmark test of the Allen Institute of Artificial Intelligence (AI2). DARPA
predicts that researchers will use manual construction, information extraction, machine
learning, crowd sourcing and other computational methods to build the knowledge
base.
3.5 Developing Cyber Resilience to Actively Address the Security
Problem of AI System
In October 2016, the USA released two strategic documents related to the development
of artificial intelligence, the National Strategic Plan for Research and Development of
Artificial Intelligence and Preparing for the Future of Artificial Intelligence. In the area
of AI security, the National AI Research and Development Strategic Plan of the USA
emphasizes the new challenges that traditional network security may bring in AI
environment [10]. It is considered that AI systems embedded in key systems must be
durable and secure to cope with large-scale deliberate network attacks. It is believed
that the application of AI system will bring some new and unique threats. At the same
time, the report also emphasizes that AI systems may eventually adopt “cyclic selfimprovement”; that is, a large number of software modifications will be made by the
software itself, rather than by human programmers. In order to ensure the security of
self-modifying systems, additional research is needed to develop self-monitoring
frameworks such as checking the consistency of the system’s behavior through the
original goal of the designer or proven value frameworks that can resist self-modifying.
In the Report “Summer Study on Autonomy” released by the Defense Science
Board of the United States Department of Defense, the answer to the cyber security
problem of intelligent military applications is to enhance the cyber resilience of
intelligent systems [4].
Cyber resilience can effectively deal with the security problems of AI system
because the focus of cyber resilience shifts from preventing cyber attacks to minimizing
the impact of cyber attacks. It does not seek to establish a perfect system to deal with all
kinds of incidents, including anti-attack. Instead, it adopts a series of technical means to
improve the system’s self-active protection ability, so that the key tasks of the system
can still be carried out normally in the “toxic carrier” environment, under attack or even
under partial control, and maintain the normal function of the system, so as to take the
next step. The measures lay the foundation [3].
In January 2013, the US Department of Defense released the report “Resilient
Military System and Advanced Cyber Threat,” which proposed the construction of
resilient military system. In August 2014, the US Air Force issued a “Cyber Resilience”
research guide, which calls for accelerating the research of resilience mechanism and
key technologies, survival and recovery of core mission functions, cyber deception,
cyber agility, resilience and agility of embedded systems [11].
412
G. Zhou