114
6 A Planetary-Scale Threat
front of a muffin will let us chose the apple more frequently. Hence, tricks like these
may be used to make us change our behavior.
Some people argue it is anyway impossible NOT to nudge people. Our environment would always influence us in subtle ways. However, I find it highly concerning,
when personal data, often collected by mass surveillance, is being used to “target”
us specifically and very effectively with personalized information.
We are well aware that our friends are able to manipulate us. Therefore, we choose
our friends carefully. Now, however, there are companies that know us better than
our friends, which can manipulate us quite effectively, without our knowledge. It is
not just Google and Facebook, which try to steer our attention, emotions, opinions,
decisions and behaviors, but also advertisement companies and others that we do not
even know by name. Due to lack of transparency, it is basically impossible to enact
our right of informational self-determination or to complain about these companies.
With the personalization of information, propaganda has become a lot more
sophisticated and effective than when the Nazis came to power in the 1930ies. People
who know that there are in relation less and less webpages and services on the Internet,
which are not personalized in some way, even speak of “The Matrix”.
39 Not only may
your news consumption be steered, but also your choice of the holiday destination
and the people you date. Humans have become the “laboratory rats” of the digital
age. Companies run millions of experiments every day to figure out how to program
our behavior ever more effectively.
For example, one of the Snowden revelations has provided insights into the JTRIG
program of the British secret service GCHQ.
40 Here, the cognitive biases of humans
41
have been mapped out, and digital ways have been developed to use them to trick us.
While most people think that such means are mainly used in psychological warfare
against enemies and secret agents, the power of Artificial Intelligence systems today
makes it possible to apply such tricks to millions or even billions of people at the
same time.
We know this from experts like Tristan Harris,
42 who has previously worked in
one of Google’s control rooms, and also from the Cambridge Analytica election
39 Tech Billionaires Convinced We Live in The Matrix Are Secretly Funding Scientists to Help
Break Us Out of It, Independent (October 6, 2016) https://www.independent.co.uk/life-style/
gadgets-and-tech/news/computer-simulation-world-matrix-scientists-elon-musk-artificial-intellige
nce-ai-a7347526.html.
40 Joint Threat Research Intelligence Group, https://en.wikipedia.org/wiki/Joint_Threat_Research_
Intelligence_Group; Controversial GCHQ Unit Engaged in Domestic Law Enforcement, Online
Propaganda, Psychological Research, The Intercept (June 22, 2015) https://theintercept.com/2015/
06/22/controversial-gchq-unit-domestic-law-enforcement-propaganda/.
41 See https://en.wikipedia.org/wiki/List_of_cognitive_biases.
42 Tristan Harris, How a handful of tech companies controls billions of minds every day, https://
www.youtube.com/watch?v=C74amJRp730 (July 28, 2017).
6 A Planetary-Scale Threat
front of a muffin will let us chose the apple more frequently. Hence, tricks like these
may be used to make us change our behavior.
Some people argue it is anyway impossible NOT to nudge people. Our environment would always influence us in subtle ways. However, I find it highly concerning,
when personal data, often collected by mass surveillance, is being used to “target”
us specifically and very effectively with personalized information.
We are well aware that our friends are able to manipulate us. Therefore, we choose
our friends carefully. Now, however, there are companies that know us better than
our friends, which can manipulate us quite effectively, without our knowledge. It is
not just Google and Facebook, which try to steer our attention, emotions, opinions,
decisions and behaviors, but also advertisement companies and others that we do not
even know by name. Due to lack of transparency, it is basically impossible to enact
our right of informational self-determination or to complain about these companies.
With the personalization of information, propaganda has become a lot more
sophisticated and effective than when the Nazis came to power in the 1930ies. People
who know that there are in relation less and less webpages and services on the Internet,
which are not personalized in some way, even speak of “The Matrix”.
39 Not only may
your news consumption be steered, but also your choice of the holiday destination
and the people you date. Humans have become the “laboratory rats” of the digital
age. Companies run millions of experiments every day to figure out how to program
our behavior ever more effectively.
For example, one of the Snowden revelations has provided insights into the JTRIG
program of the British secret service GCHQ.
40 Here, the cognitive biases of humans
41
have been mapped out, and digital ways have been developed to use them to trick us.
While most people think that such means are mainly used in psychological warfare
against enemies and secret agents, the power of Artificial Intelligence systems today
makes it possible to apply such tricks to millions or even billions of people at the
same time.
We know this from experts like Tristan Harris,
42 who has previously worked in
one of Google’s control rooms, and also from the Cambridge Analytica election
39 Tech Billionaires Convinced We Live in The Matrix Are Secretly Funding Scientists to Help
Break Us Out of It, Independent (October 6, 2016) https://www.independent.co.uk/life-style/
gadgets-and-tech/news/computer-simulation-world-matrix-scientists-elon-musk-artificial-intellige
nce-ai-a7347526.html.
40 Joint Threat Research Intelligence Group, https://en.wikipedia.org/wiki/Joint_Threat_Research_
Intelligence_Group; Controversial GCHQ Unit Engaged in Domestic Law Enforcement, Online
Propaganda, Psychological Research, The Intercept (June 22, 2015) https://theintercept.com/2015/
06/22/controversial-gchq-unit-domestic-law-enforcement-propaganda/.
41 See https://en.wikipedia.org/wiki/List_of_cognitive_biases.
42 Tristan Harris, How a handful of tech companies controls billions of minds every day, https://
www.youtube.com/watch?v=C74amJRp730 (July 28, 2017).