120
I. T. Freire et al.
However, what is still missing in the field of AI is a solid solution that would
integrate the three decision-making systems identified by cognitive neuroscience
into one single architecture. If we aspire to model human moral decision-making
with its complex nuances arising from the interplay between the three processes,
as well as to implement them in a robotic system, we have to use a theoretical
framework that is able to accommodate them into a single cognitive architecture.
To address that issue, we draw upon the Distributed Adaptive Control theory
of mind and brain that proposes that cognition is based on four control layers
operating at different levels of abstraction [38–40]. The first level, the Soma layer,
contains the whole body of the agent with all the sensors and actuators and
represents the interface between the agent and its environment. The Reactive
layer integrates the Pavlovian decision-making system that regulates the fast
and unconditioned reactions to favorable and averse stimuli, implemented as
self-regulated sensorimotor control loops. These reactive interactions with the
environment bootstrap the learning of model-free relationships in the Adaptive
layer allowing the acquisition of a state-action space of the agent-environment
interaction. The Contextual layer acquires temporally extended model-based
policies that contribute to the acquisition of more abstract cognitive abilities
such as goal selection, memory and planning. These higher-level representations,
in turn, affect the behavior of lower layers in a top-down fashion. Control in this
architecture is therefore distributed between all layers through interactions in
both directions, top-down and bottom-up, as well as laterally within each layer.
The three layers of the Distributed Adaptive Control architecture can be
mapped to biological systems as different learning processes operating at different timescales, as recently proposed in [8]. The reactive layer implementing the
Pavlovian decision-making system would correspond to adaptation to the environment through slow evolutionary processes. The adaptive layer would then
comprise learned behaviors of the model-free decision system, acquired through
development. Finally, model-based learning in the contextual layer would happen
at the diurnal timescale.
Shaping of cognition through the lens of evolution, development and learning brings us to an apparent contradiction. Paradoxically, the control processes
subject to the fastest timescales of adaptation and learning (contextual, modelbased) are the ones responsible for providing slow and deliberate decisions. Conversely, the more primitive control mechanisms subject to the slowest forms of
adaptation (reactive, Pavlovian) are the ones responsible for producing fast and
robust decisions. From a control perspective, however, this observation makes
a lot of sense. One the one hand, a reactive system that provides robust and
reliable responses when its needed will be intrinsically slow in adapting (robustness), but extremely useful if it is fast in delivering responses towards the most
dangerous or vital things of the environment. On the other hand, a system that
provides flexible responses to the ever-changing details of the its surroundings
might give an organism the necessary advantage against its competitors. Of
course, such attention to the details implies a lot of information to take into
account, therefore its flexibility comes at a cost in response or processing time.
I. T. Freire et al.
However, what is still missing in the field of AI is a solid solution that would
integrate the three decision-making systems identified by cognitive neuroscience
into one single architecture. If we aspire to model human moral decision-making
with its complex nuances arising from the interplay between the three processes,
as well as to implement them in a robotic system, we have to use a theoretical
framework that is able to accommodate them into a single cognitive architecture.
To address that issue, we draw upon the Distributed Adaptive Control theory
of mind and brain that proposes that cognition is based on four control layers
operating at different levels of abstraction [38–40]. The first level, the Soma layer,
contains the whole body of the agent with all the sensors and actuators and
represents the interface between the agent and its environment. The Reactive
layer integrates the Pavlovian decision-making system that regulates the fast
and unconditioned reactions to favorable and averse stimuli, implemented as
self-regulated sensorimotor control loops. These reactive interactions with the
environment bootstrap the learning of model-free relationships in the Adaptive
layer allowing the acquisition of a state-action space of the agent-environment
interaction. The Contextual layer acquires temporally extended model-based
policies that contribute to the acquisition of more abstract cognitive abilities
such as goal selection, memory and planning. These higher-level representations,
in turn, affect the behavior of lower layers in a top-down fashion. Control in this
architecture is therefore distributed between all layers through interactions in
both directions, top-down and bottom-up, as well as laterally within each layer.
The three layers of the Distributed Adaptive Control architecture can be
mapped to biological systems as different learning processes operating at different timescales, as recently proposed in [8]. The reactive layer implementing the
Pavlovian decision-making system would correspond to adaptation to the environment through slow evolutionary processes. The adaptive layer would then
comprise learned behaviors of the model-free decision system, acquired through
development. Finally, model-based learning in the contextual layer would happen
at the diurnal timescale.
Shaping of cognition through the lens of evolution, development and learning brings us to an apparent contradiction. Paradoxically, the control processes
subject to the fastest timescales of adaptation and learning (contextual, modelbased) are the ones responsible for providing slow and deliberate decisions. Conversely, the more primitive control mechanisms subject to the slowest forms of
adaptation (reactive, Pavlovian) are the ones responsible for producing fast and
robust decisions. From a control perspective, however, this observation makes
a lot of sense. One the one hand, a reactive system that provides robust and
reliable responses when its needed will be intrinsically slow in adapting (robustness), but extremely useful if it is fast in delivering responses towards the most
dangerous or vital things of the environment. On the other hand, a system that
provides flexible responses to the ever-changing details of the its surroundings
might give an organism the necessary advantage against its competitors. Of
course, such attention to the details implies a lot of information to take into
account, therefore its flexibility comes at a cost in response or processing time.