4.3 The Digital “Magic Wand”: A Remote Control for Humans?
67
to us in a way that is made to manipulate our choices. Personalized ads do this
every day. They let us buy certain products or services that we wouldn’t have cared
about otherwise. The more personal data is collected about us, the more efficient do
personalized ads get.
It would be surprising if such possibilities to manipulate our choices weren’t
attractive to politicians, too. Given the overwhelming amount of data available today,
information needs to be filtered to be useful for us. Such filtering will inevitably be
done in the interests of those who do the filtering or pay for it. A recent Facebook
experiment involving almost 700,000 users showed that it is, in fact, possible to
manipulate people’s feelings and moods.
8
Therefore, will it soon be possible to turn “omniscience” into “omnipotence”? In
other words, could those who have access to all the data finally control everything?
Let’s call the hypothetical tool creating such power a digital “magic wand”. Suppose
again you were the president of a country and had such a magic wand. Could you be
“omnibenevolent”, i.e. take the best decisions for society and every single one of us?
Many people might say that forecasting societal trends is different from forecasting
the weather. While the weather will not react to a forecast, people will. This seems to
suggest that societal predictions will be unreliable (or even “self-defeating prophecies”), if the predictions are published. But what if a secret institution evaluated our
data and advised the government about the right decisions to be made? The magic
wand would then be used to manipulate people’s behavior based on the evidence
provided by the crystal ball. Could such a scheme work?
4.4 A New World Order Based on Information?
A wishful “benevolent dictator” or “wise king” would probably see the crystal ball
and the magic wand as perfect tools to create the desired societal outcomes. Of
course, the “wise king” could not fulfill our wishes all the time, but he might be able
to create better outcomes on average for everyone as long as we obey him. So, a “wise
king” would sometimes interfere with our individual freedom, if the autonomous
decisions of people would otherwise obstruct his plan. This, however, would result
in a form of totalitarian technocracy, where all of us would have to follow personalized
instructions, as if they were commands from God. One might distinguish two kinds
of such “big government” approaches. The “Big Brother” society would build on
mass surveillance and the punishment of people, who don’t obey the commands or
are unlikely to obey them as determined by “predictive policing” algorithms (even
though the effectiveness is questionable
9 ). The “Big Manipulator” society would
8 Kramer et al. [3]. This experiment created a big «shit storm»: http://www.wsj.com/articles/furorerupts-over-facebook-experiment-on-users-1404085840.
9 Journalist’s Resource (6 November 2014), The effectiveness of predictive policing: Lessons from a
randomized controlled trial, https://journalistsresource.org/studies/government/criminal-justice/pre
dictive-policing-randomized-controlledtrial/. ZEIT Online (29 March 2015), Predictive Policing—
Noch hat niemand bewiesen, dass Data Mining der Polizei hilft, http://www.zeit.de/digital/datens
chutz/2015-03/predictive-policing-software-polizei-precobs.
67
to us in a way that is made to manipulate our choices. Personalized ads do this
every day. They let us buy certain products or services that we wouldn’t have cared
about otherwise. The more personal data is collected about us, the more efficient do
personalized ads get.
It would be surprising if such possibilities to manipulate our choices weren’t
attractive to politicians, too. Given the overwhelming amount of data available today,
information needs to be filtered to be useful for us. Such filtering will inevitably be
done in the interests of those who do the filtering or pay for it. A recent Facebook
experiment involving almost 700,000 users showed that it is, in fact, possible to
manipulate people’s feelings and moods.
8
Therefore, will it soon be possible to turn “omniscience” into “omnipotence”? In
other words, could those who have access to all the data finally control everything?
Let’s call the hypothetical tool creating such power a digital “magic wand”. Suppose
again you were the president of a country and had such a magic wand. Could you be
“omnibenevolent”, i.e. take the best decisions for society and every single one of us?
Many people might say that forecasting societal trends is different from forecasting
the weather. While the weather will not react to a forecast, people will. This seems to
suggest that societal predictions will be unreliable (or even “self-defeating prophecies”), if the predictions are published. But what if a secret institution evaluated our
data and advised the government about the right decisions to be made? The magic
wand would then be used to manipulate people’s behavior based on the evidence
provided by the crystal ball. Could such a scheme work?
4.4 A New World Order Based on Information?
A wishful “benevolent dictator” or “wise king” would probably see the crystal ball
and the magic wand as perfect tools to create the desired societal outcomes. Of
course, the “wise king” could not fulfill our wishes all the time, but he might be able
to create better outcomes on average for everyone as long as we obey him. So, a “wise
king” would sometimes interfere with our individual freedom, if the autonomous
decisions of people would otherwise obstruct his plan. This, however, would result
in a form of totalitarian technocracy, where all of us would have to follow personalized
instructions, as if they were commands from God. One might distinguish two kinds
of such “big government” approaches. The “Big Brother” society would build on
mass surveillance and the punishment of people, who don’t obey the commands or
are unlikely to obey them as determined by “predictive policing” algorithms (even
though the effectiveness is questionable
9 ). The “Big Manipulator” society would
8 Kramer et al. [3]. This experiment created a big «shit storm»: http://www.wsj.com/articles/furorerupts-over-facebook-experiment-on-users-1404085840.
9 Journalist’s Resource (6 November 2014), The effectiveness of predictive policing: Lessons from a
randomized controlled trial, https://journalistsresource.org/studies/government/criminal-justice/pre
dictive-policing-randomized-controlledtrial/. ZEIT Online (29 March 2015), Predictive Policing—
Noch hat niemand bewiesen, dass Data Mining der Polizei hilft, http://www.zeit.de/digital/datens
chutz/2015-03/predictive-policing-software-polizei-precobs.