14.2 Can We Trust It?
299
14.2.8 Learning to Die?
Indices based on multiple measurements or data sets are also being used to recommend prison sentences,
56 to advise on whether a certain medical operation is
economic, given the overall health and age of a person, or even to make life-ordeath decisions.
57 One scenario that has recently been obsessively discussed is the
so-called “trolley problem”.
58 Here, an extremely rare situation is assumed, where
an autonomous vehicle cannot brake quickly enough, and one person or another
will die (e.g. one pedestrian or another one, or the car driver). Given that, thanks
to camera technology, pattern recognition and Big Data, a future car might distinguish between age, race, gender, social status etc., controversial questions arise.
For example, should the car save a grandmother, while sacrificing an unemployed
person, or vice versa? Note that such considerations are not in accordance with the
equality principle of democratic constitutions
59 and also not compatible with classical medical ethics.
60 Nevertheless, questions like these have been recently raised in
what was framed as the “Moral Machine Experiment”,
61 and recent organ transplant
practices already apply new policies considering previous lifestyle.
62
Now, assume that we would put such data-based life-and-death situations into
laws. Can you imagine what will happen if such principles originally intended to
save human lives would be applied in possible future scenarios characterized by
scarce resources? Then, the algorithm would turn into a killer algorithm bringing a
“digital holocaust” on the way, which might sort out old and ill people, and probably
also people of low social status, including certain minorities. This is the true danger
of creating autonomous systems that may seriously interfere with our lives.
63 It does
not take killer robots for autonomous systems to become a threat to our lives, if the
supply of resources falls short.
56 AI is convicting criminals and determining jail time, but is it fair? World Economic Forum
(November 19, 2018) https://www.weforum.org/agenda/2018/11/algorithms-court-criminals-jailtime-fair/; AI is sending people to jail—and getting it wrong, MIT Technology Review (January
21, 2019) https://www.technologyreview.com/s/612775/algorithms-criminal-justice-ai/.
57 Wenn Software über Leben und Tod entscheidet, ZDF (December 20, 2017) https://web.arc
hive.org/web/20171222135350/, https://www.zdf.de/nachrichten/heute/software-soll-ueber-lebenund-tod-entscheiden-100.html.
58 Nagler et al. [13].
59 Automatisiertes und vernetztes Fahren, Bericht der Ethikkommission, Bundesministerium für
Verkehr und digitale Infrastruktur, https://www.bmvi.de/SharedDocs/DE/Publikationen/DG/ber
icht-der-ethik-kommission.pdf.
60 WMA Declaration of Geneva: The Physician’s Pledge, World Medical Association, https://www.
wma.net/policies-post/wma-declaration-of-geneva/.
61 Awad et al. [14].
62 Wer erhält die Organspende? Swiss Transplant https://www.swisstransplant.org/de/organspendetransplantation/rund-umsspenden/wer-erhaelt-die-spende/ (accessed August 8, 2020).
63 Artificial Intelligence: Lethal Autonomous Weapons Systems and Peace Time Threats
(November 28, 2017) https://ict4peace.org/activities/artificial-intelligence-lethal-autonomous-wea
pons-systems-and-peace-time-threats/.
299
14.2.8 Learning to Die?
Indices based on multiple measurements or data sets are also being used to recommend prison sentences,
56 to advise on whether a certain medical operation is
economic, given the overall health and age of a person, or even to make life-ordeath decisions.
57 One scenario that has recently been obsessively discussed is the
so-called “trolley problem”.
58 Here, an extremely rare situation is assumed, where
an autonomous vehicle cannot brake quickly enough, and one person or another
will die (e.g. one pedestrian or another one, or the car driver). Given that, thanks
to camera technology, pattern recognition and Big Data, a future car might distinguish between age, race, gender, social status etc., controversial questions arise.
For example, should the car save a grandmother, while sacrificing an unemployed
person, or vice versa? Note that such considerations are not in accordance with the
equality principle of democratic constitutions
59 and also not compatible with classical medical ethics.
60 Nevertheless, questions like these have been recently raised in
what was framed as the “Moral Machine Experiment”,
61 and recent organ transplant
practices already apply new policies considering previous lifestyle.
62
Now, assume that we would put such data-based life-and-death situations into
laws. Can you imagine what will happen if such principles originally intended to
save human lives would be applied in possible future scenarios characterized by
scarce resources? Then, the algorithm would turn into a killer algorithm bringing a
“digital holocaust” on the way, which might sort out old and ill people, and probably
also people of low social status, including certain minorities. This is the true danger
of creating autonomous systems that may seriously interfere with our lives.
63 It does
not take killer robots for autonomous systems to become a threat to our lives, if the
supply of resources falls short.
56 AI is convicting criminals and determining jail time, but is it fair? World Economic Forum
(November 19, 2018) https://www.weforum.org/agenda/2018/11/algorithms-court-criminals-jailtime-fair/; AI is sending people to jail—and getting it wrong, MIT Technology Review (January
21, 2019) https://www.technologyreview.com/s/612775/algorithms-criminal-justice-ai/.
57 Wenn Software über Leben und Tod entscheidet, ZDF (December 20, 2017) https://web.arc
hive.org/web/20171222135350/, https://www.zdf.de/nachrichten/heute/software-soll-ueber-lebenund-tod-entscheiden-100.html.
58 Nagler et al. [13].
59 Automatisiertes und vernetztes Fahren, Bericht der Ethikkommission, Bundesministerium für
Verkehr und digitale Infrastruktur, https://www.bmvi.de/SharedDocs/DE/Publikationen/DG/ber
icht-der-ethik-kommission.pdf.
60 WMA Declaration of Geneva: The Physician’s Pledge, World Medical Association, https://www.
wma.net/policies-post/wma-declaration-of-geneva/.
61 Awad et al. [14].
62 Wer erhält die Organspende? Swiss Transplant https://www.swisstransplant.org/de/organspendetransplantation/rund-umsspenden/wer-erhaelt-die-spende/ (accessed August 8, 2020).
63 Artificial Intelligence: Lethal Autonomous Weapons Systems and Peace Time Threats
(November 28, 2017) https://ict4peace.org/activities/artificial-intelligence-lethal-autonomous-wea
pons-systems-and-peace-time-threats/.