Machine Morality
117
Although theoretical accounts in this field, as with the majority of eternal
questions, tend to fall in the nature-nurture debate, a synthetic approach states
that moral reasoning arises from complex social decision-making and involves
both unconscious and deliberate processes [1,12,20,37,46].
But what are the mechanisms behind our impressive moral capacities? Is it
our ability to make decisions taking others into account? Or is it our ability to
predict internal states of others? Can we obtain a better understanding of this
phenomenon to contribute to building more cooperative and caring societies?
Can we design living machines that can differentiate between what is right and
what is wrong? What are the computational mechanisms behind our sense of
morality and can it be reproduced in artificial systems? We will try to address
those questions through the cross-field integration of insights from philosophy,
social cognitive neuroscience, evolutionary anthropology, artificial intelligence
and robotics.
In the philosophical tradition, there are two main standpoints that propose
different principles behind our moral decision-making. Utilitarianism assumes
that morality implies maximizing the total amount of “utility” (a measure of
happiness) in the world [27]. Since utilitarians evaluate actions based on their
outcomes, their views are called “consequentialist”. Deontology, on the other
hand, evaluates the actions themselves as being morally justified or not, regardless of their consequences [24]. In the field of artificial intelligence (AI) there
have been attempts to implement both of those normative views on morality in
artificial systems [43]. The implementations of moral agents are employing two
broad approaches: the top-down application of ethical theories, and the bottomup construction of systems that aim at specified goals or standards which may
or may not be specified in explicit theoretical terms [44,45].
A bottom-up approach to ethics in AI addresses norms as being an integral part of the activity of agents rather than explicitly formulated in terms
of a general theory [42]. Complex bottom-up architectures excel in their ability
to dynamically integrate input from various sources, as demonstrated in their
successful application for sensorimotor control in robotics. The reverse side of
this coin is usually seen in failures to understand which goals have to be used
to assess choices and actions in dynamically changing conditions. Bottom-up
agents are most efficient when they are aimed at fulfilling one distinct objective.
With multiple goals or confusing sources of information, shaping a clear scenario
becomes a challenging task for bottom-up systems [28].
A top-down approach, on the other hand, utilizes the explicitly specified
ethical theory and analyzes the resulting computational requirements in order
to develop algorithms and agents capable of implementing that theory. These
systems can provide clear solutions to moral dilemmas by directly applying the
norms specified in their implementation. However, problems intrinsic to consequentalist and deontological approaches will manifest in terms of computational resources. In the case of consequentalist approaches, utility calculations
of possible outcomes can scale dramatically, and deontological assessment of the
117
Although theoretical accounts in this field, as with the majority of eternal
questions, tend to fall in the nature-nurture debate, a synthetic approach states
that moral reasoning arises from complex social decision-making and involves
both unconscious and deliberate processes [1,12,20,37,46].
But what are the mechanisms behind our impressive moral capacities? Is it
our ability to make decisions taking others into account? Or is it our ability to
predict internal states of others? Can we obtain a better understanding of this
phenomenon to contribute to building more cooperative and caring societies?
Can we design living machines that can differentiate between what is right and
what is wrong? What are the computational mechanisms behind our sense of
morality and can it be reproduced in artificial systems? We will try to address
those questions through the cross-field integration of insights from philosophy,
social cognitive neuroscience, evolutionary anthropology, artificial intelligence
and robotics.
In the philosophical tradition, there are two main standpoints that propose
different principles behind our moral decision-making. Utilitarianism assumes
that morality implies maximizing the total amount of “utility” (a measure of
happiness) in the world [27]. Since utilitarians evaluate actions based on their
outcomes, their views are called “consequentialist”. Deontology, on the other
hand, evaluates the actions themselves as being morally justified or not, regardless of their consequences [24]. In the field of artificial intelligence (AI) there
have been attempts to implement both of those normative views on morality in
artificial systems [43]. The implementations of moral agents are employing two
broad approaches: the top-down application of ethical theories, and the bottomup construction of systems that aim at specified goals or standards which may
or may not be specified in explicit theoretical terms [44,45].
A bottom-up approach to ethics in AI addresses norms as being an integral part of the activity of agents rather than explicitly formulated in terms
of a general theory [42]. Complex bottom-up architectures excel in their ability
to dynamically integrate input from various sources, as demonstrated in their
successful application for sensorimotor control in robotics. The reverse side of
this coin is usually seen in failures to understand which goals have to be used
to assess choices and actions in dynamically changing conditions. Bottom-up
agents are most efficient when they are aimed at fulfilling one distinct objective.
With multiple goals or confusing sources of information, shaping a clear scenario
becomes a challenging task for bottom-up systems [28].
A top-down approach, on the other hand, utilizes the explicitly specified
ethical theory and analyzes the resulting computational requirements in order
to develop algorithms and agents capable of implementing that theory. These
systems can provide clear solutions to moral dilemmas by directly applying the
norms specified in their implementation. However, problems intrinsic to consequentalist and deontological approaches will manifest in terms of computational resources. In the case of consequentalist approaches, utility calculations
of possible outcomes can scale dramatically, and deontological assessment of the