12.1 The “Benevolent Dictator” Is Dead
251
West use it, or some other “super-score”
12 —a one-dimensional score created from a
multitude of measurements). This score would determine our value for society, and
it would determine our rights and opportunities.
According to its engineers, the superintelligent system would act like a “benevolent dictator”.
13 Even though the above storyline sounds kind of plausible, it contains
a couple of serious mistakes:
1. If you want to optimize the world, you would need to choose a goal function.
Unfortunately, we don’t know, what is the right goal function for the world. There
is not even a science I know of, which would tell us—on scientific grounds—
what goal function to choose. Should it be profit (GDP per capita)? Should it be
sustainability (if yes, how to define it)? Should it be life expectancy? Should it
be happiness? Or what else should it be?
2. In the past years, the economic players tried to maximize profit. This, however,
has brought our planet to the brink of disaster—so much so that we are now talking
about a “climate emergency”. Suppose we would now tell a superintelligent
system to maximize sustainability. Then, it could easily happen that the system
would suggest to end the lives of many people—or if it was equipped with tools
for this (such as Skynet seems to be
14 ), it might even put some people to death.
3. I actually doubt that optimization is the right approach, even though “optimization” sounds like a good thing. The reason is as follows: In order to optimize,
one needs a one-dimensional goal function, otherwise one cannot decide what
solution is better and which one is worse. (Such decisions need “>” and “<”
operations.) Hence, the complexity of the system that should be optimized needs
to be projected on one dimension. In many cases, this will lead to terrible oversimplifications. In any case, once you have decided for the goal, you would push
all other goals into the background, and after some time of neglecting these
goals, say, 50 years later, one of them will have become an even bigger problem
to solve. Therefore, in our society, we cannot afford to focus on one goal and push
back all others, in contrast to what a company might do. For a society to thrive,
one needs to pursue various goals at the same time, and find a suitable balance
between these goals. That, however, requires pluralism and diversity, not global
centralized optimization. In the next chapter, we will discuss how this may be
done.
4. Even if it was the right thing to maximize a particular goal function, one would
need to rely on the correctness of the outcome. However, algorithms (machine
learning and others) do not always converge.
15 They may imply biases and
12 Superscoring: Wie wertvoll sind Sie für die Gesellschaft? PC Welt (February 5, 2020) https://www.
pcwelt.de/ratgeber/Superscoring-Wie-wertvoll-sind-Sie-fuer-die-Gesellschaft-10633488.html.
13 See https://en.wikipedia.org/wiki/Benevolent_dictatorship.
14 The NSA’s SKYNET program may be killing thousands of innocent people, Ars Technica
(February 16, 2016) https://arstechnica.com/information-technology/2016/02/the-nsas-skynet-pro
gram-may-be-killing-thousands-of-innocent-people/.
15 Particularly if the underlying data have scale-free characteristics.
251
West use it, or some other “super-score”
12 —a one-dimensional score created from a
multitude of measurements). This score would determine our value for society, and
it would determine our rights and opportunities.
According to its engineers, the superintelligent system would act like a “benevolent dictator”.
13 Even though the above storyline sounds kind of plausible, it contains
a couple of serious mistakes:
1. If you want to optimize the world, you would need to choose a goal function.
Unfortunately, we don’t know, what is the right goal function for the world. There
is not even a science I know of, which would tell us—on scientific grounds—
what goal function to choose. Should it be profit (GDP per capita)? Should it be
sustainability (if yes, how to define it)? Should it be life expectancy? Should it
be happiness? Or what else should it be?
2. In the past years, the economic players tried to maximize profit. This, however,
has brought our planet to the brink of disaster—so much so that we are now talking
about a “climate emergency”. Suppose we would now tell a superintelligent
system to maximize sustainability. Then, it could easily happen that the system
would suggest to end the lives of many people—or if it was equipped with tools
for this (such as Skynet seems to be
14 ), it might even put some people to death.
3. I actually doubt that optimization is the right approach, even though “optimization” sounds like a good thing. The reason is as follows: In order to optimize,
one needs a one-dimensional goal function, otherwise one cannot decide what
solution is better and which one is worse. (Such decisions need “>” and “<”
operations.) Hence, the complexity of the system that should be optimized needs
to be projected on one dimension. In many cases, this will lead to terrible oversimplifications. In any case, once you have decided for the goal, you would push
all other goals into the background, and after some time of neglecting these
goals, say, 50 years later, one of them will have become an even bigger problem
to solve. Therefore, in our society, we cannot afford to focus on one goal and push
back all others, in contrast to what a company might do. For a society to thrive,
one needs to pursue various goals at the same time, and find a suitable balance
between these goals. That, however, requires pluralism and diversity, not global
centralized optimization. In the next chapter, we will discuss how this may be
done.
4. Even if it was the right thing to maximize a particular goal function, one would
need to rely on the correctness of the outcome. However, algorithms (machine
learning and others) do not always converge.
15 They may imply biases and
12 Superscoring: Wie wertvoll sind Sie für die Gesellschaft? PC Welt (February 5, 2020) https://www.
pcwelt.de/ratgeber/Superscoring-Wie-wertvoll-sind-Sie-fuer-die-Gesellschaft-10633488.html.
13 See https://en.wikipedia.org/wiki/Benevolent_dictatorship.
14 The NSA’s SKYNET program may be killing thousands of innocent people, Ars Technica
(February 16, 2016) https://arstechnica.com/information-technology/2016/02/the-nsas-skynet-pro
gram-may-be-killing-thousands-of-innocent-people/.
15 Particularly if the underlying data have scale-free characteristics.