124
I. T. Freire et al.
assembling/disassembling components in a tabletop setting. It is also important to mention that model-free learning is not a part of this implementation,
since the requirements of learning time and context demanded a more robust
solution. Therefore, our moral cognitive architecture will operate using only
the Reactive and Contextual layers. However, Machine-Morality-as-Cooperation
approach promises functional benefits even with these limitations, as it aims at
promoting cooperation through putting human worker always first. Promoting
cooperation is achieved through optimizing different pre-determined parameters
at the reactive level, such as, for instance, speed of operation, distance to human
worker and number of tasks allocated to the robot. The target values of those
parameters are mapped to actual human preferences obtained from a worker
model built in the contextual layer of the architecture. Thanks to this feature,
each human worker is seen as a unique context. Contextual layer also comprises
information regarding tool use and specifics of each ongoing mechanical task
performed. Thus the reactive control systems in charge of harm avoidance is
also contextually aware through the regulation of situational parameters such as
safe distance, affordances of the object that might pose risks, etc. This dynamic
contextual modulation of the target values of each reactive control system is also
known as “allostatic control”, and has been shown to produce adaptive behavior
in both living [35] and artificial [41] systems.
7 Discussion
In this paper, we present a novel computational framework for modeling moral
decision-making in artificial agents (Machine-Morality-as-Cooperation) that
integrates recent advances from the cross-disciplinary moral decision-making literature into a single architecture. We build upon previous work outlining the
cognitive elements that an artificial agent would need for exhibiting latent morality [2], and we extend it by providing a first computational description of the
cognitive architecture of such agent.
Recent evidence from the field of cognitive neuroscience describes the key
roles of three different decision-making processes (Pavlovian, model-free, modelbased) that are defined by distinct neural substrates in the human brain. We
have shown how computational models of these three cognitive processes can
be implemented in a single cognitive architecture by using the distributed and
hierarchical organization proposed by the DAC theoretical framework. Moreover,
we propose that a pro-social drive to cooperate exists at the Pavlovian level that
can also bias the rest of the decision systems, thus extending current state-ofthe-art descriptive models based on harm-aversion.
We believe that the key to advancing our understanding of human morality
lies in comprehensive theoretical synthesis across multiple fields that study this
phenomenon. The pursuit of implementing morality in living machines will not
only affect how we perceive and interact with future autonomous agents: it will
help us get a more profound vision of the human cognition. In this view, the
I. T. Freire et al.
assembling/disassembling components in a tabletop setting. It is also important to mention that model-free learning is not a part of this implementation,
since the requirements of learning time and context demanded a more robust
solution. Therefore, our moral cognitive architecture will operate using only
the Reactive and Contextual layers. However, Machine-Morality-as-Cooperation
approach promises functional benefits even with these limitations, as it aims at
promoting cooperation through putting human worker always first. Promoting
cooperation is achieved through optimizing different pre-determined parameters
at the reactive level, such as, for instance, speed of operation, distance to human
worker and number of tasks allocated to the robot. The target values of those
parameters are mapped to actual human preferences obtained from a worker
model built in the contextual layer of the architecture. Thanks to this feature,
each human worker is seen as a unique context. Contextual layer also comprises
information regarding tool use and specifics of each ongoing mechanical task
performed. Thus the reactive control systems in charge of harm avoidance is
also contextually aware through the regulation of situational parameters such as
safe distance, affordances of the object that might pose risks, etc. This dynamic
contextual modulation of the target values of each reactive control system is also
known as “allostatic control”, and has been shown to produce adaptive behavior
in both living [35] and artificial [41] systems.
7 Discussion
In this paper, we present a novel computational framework for modeling moral
decision-making in artificial agents (Machine-Morality-as-Cooperation) that
integrates recent advances from the cross-disciplinary moral decision-making literature into a single architecture. We build upon previous work outlining the
cognitive elements that an artificial agent would need for exhibiting latent morality [2], and we extend it by providing a first computational description of the
cognitive architecture of such agent.
Recent evidence from the field of cognitive neuroscience describes the key
roles of three different decision-making processes (Pavlovian, model-free, modelbased) that are defined by distinct neural substrates in the human brain. We
have shown how computational models of these three cognitive processes can
be implemented in a single cognitive architecture by using the distributed and
hierarchical organization proposed by the DAC theoretical framework. Moreover,
we propose that a pro-social drive to cooperate exists at the Pavlovian level that
can also bias the rest of the decision systems, thus extending current state-ofthe-art descriptive models based on harm-aversion.
We believe that the key to advancing our understanding of human morality
lies in comprehensive theoretical synthesis across multiple fields that study this
phenomenon. The pursuit of implementing morality in living machines will not
only affect how we perceive and interact with future autonomous agents: it will
help us get a more profound vision of the human cognition. In this view, the