296
14 Summary: What’s Wrong with AI?
other words, individuals would be represented by one number, which is often referred
to as “Citizen Score”.
45 This appears to be done, for example, by the “Karma Police
Program”,
46 which was revealed on September 25, 2015—the day on which Pope
Francis demanded the Agenda 2030, a set of 17 Sustainable Development Goals,
in front of the UN General Assembly.
47 China is currently testing a Citizen Score
approach named “Social Credit Score”.
48 The program may be seen as an attempt to
make citizens obedient to the government’s wishes. This has been criticized as data
dictatorship
49 or technological totalitarianism.
50
A Citizen Score establishes, in principle, a Big-Data-driven neo-feudalistic
order.
51 Those with a high score will get access to good offers, products, and services,
and basically get anything they desire. Those with a low score may lose their human
rights and may be deprived from certain opportunities. This may include such things
as travel visa to other countries, job opportunities, the allowance to use a plane or
fast train, the Internet speed, or certain kinds of medical treatment. In other words,
high-score citizens will live kind of “in heaven”, while a large number of low-score
citizens may experience something like “hell on Earth”. It’s something like a digital
judgment day scenario, where you get negative points, when you cross a red pedestrian traffic light, if you read critical political news, or if you have friends who read
critical political news, to give some realistic examples.
The idea behind this is to establish “total justice” (in the language of the Agenda
2030: “strong institutions”), particularly for situations when societies are faced with
scarce resources. However, a Citizen Score will not do justice to people at all, as these
are different by nature. All the arbitrariness of a superscore lies in the weights of the
different measurements that go into the underlying index.
52 These weights will be to
45 Big data meets Big Brother as China moves to rate ist citizens, Wired (October 21. 2017) https://
www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion.
46 Profiled: From Radio to Porn, British Spies Track Web Users’ Online Identities, The Intercept
(September 25, 2015) https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-usersonline-identities/.
47 Sustainable development summit 2015: World leaders agree sustainable development goals—as
it happened, The Guardian (September 25, 2015) https://www.theguardian.com/global-develo
pment/live/2015/sep/25/un-sustainable-development-summit-2015-goals-sdgs-united-nations-gen
eral-assembly-70th-session-new-york-live.
48 China has started ranking citizens with a creepy ‘social credit’ system … Business Insider (October
29, 2018) https://www.businessinsider.com/china-social-credit-system-punishments-and-rewardsexplained-2018-4.
49 Das Digital-Manifest: Digitale Demokratie statt Datendiktatur, Spektrum der Wissenschaft
(November 12, 2015) https://www.spektrum.de/thema/das-digital-manifest/1375924, English
translation: Will Democracy Survive Big Data and Artificial Intelligence? (February 25,
2017) https://www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificialintelligence/.
50 Schirrmacher [11].
51 Superscoring: Wie wertvoll sind Sie für die Gesellschaft? PC Welt (February 5, 2020) https://
www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificial-intelligence/.
52 https://www.pcwelt.de/ratgeber/Superscoring-Wie-wertvoll-sind-Sie-fuer-die-Gesellschaft-106
33488.html.
14 Summary: What’s Wrong with AI?
other words, individuals would be represented by one number, which is often referred
to as “Citizen Score”.
45 This appears to be done, for example, by the “Karma Police
Program”,
46 which was revealed on September 25, 2015—the day on which Pope
Francis demanded the Agenda 2030, a set of 17 Sustainable Development Goals,
in front of the UN General Assembly.
47 China is currently testing a Citizen Score
approach named “Social Credit Score”.
48 The program may be seen as an attempt to
make citizens obedient to the government’s wishes. This has been criticized as data
dictatorship
49 or technological totalitarianism.
50
A Citizen Score establishes, in principle, a Big-Data-driven neo-feudalistic
order.
51 Those with a high score will get access to good offers, products, and services,
and basically get anything they desire. Those with a low score may lose their human
rights and may be deprived from certain opportunities. This may include such things
as travel visa to other countries, job opportunities, the allowance to use a plane or
fast train, the Internet speed, or certain kinds of medical treatment. In other words,
high-score citizens will live kind of “in heaven”, while a large number of low-score
citizens may experience something like “hell on Earth”. It’s something like a digital
judgment day scenario, where you get negative points, when you cross a red pedestrian traffic light, if you read critical political news, or if you have friends who read
critical political news, to give some realistic examples.
The idea behind this is to establish “total justice” (in the language of the Agenda
2030: “strong institutions”), particularly for situations when societies are faced with
scarce resources. However, a Citizen Score will not do justice to people at all, as these
are different by nature. All the arbitrariness of a superscore lies in the weights of the
different measurements that go into the underlying index.
52 These weights will be to
45 Big data meets Big Brother as China moves to rate ist citizens, Wired (October 21. 2017) https://
www.wired.co.uk/article/chinese-government-social-credit-score-privacy-invasion.
46 Profiled: From Radio to Porn, British Spies Track Web Users’ Online Identities, The Intercept
(September 25, 2015) https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-usersonline-identities/.
47 Sustainable development summit 2015: World leaders agree sustainable development goals—as
it happened, The Guardian (September 25, 2015) https://www.theguardian.com/global-develo
pment/live/2015/sep/25/un-sustainable-development-summit-2015-goals-sdgs-united-nations-gen
eral-assembly-70th-session-new-york-live.
48 China has started ranking citizens with a creepy ‘social credit’ system … Business Insider (October
29, 2018) https://www.businessinsider.com/china-social-credit-system-punishments-and-rewardsexplained-2018-4.
49 Das Digital-Manifest: Digitale Demokratie statt Datendiktatur, Spektrum der Wissenschaft
(November 12, 2015) https://www.spektrum.de/thema/das-digital-manifest/1375924, English
translation: Will Democracy Survive Big Data and Artificial Intelligence? (February 25,
2017) https://www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificialintelligence/.
50 Schirrmacher [11].
51 Superscoring: Wie wertvoll sind Sie für die Gesellschaft? PC Welt (February 5, 2020) https://
www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificial-intelligence/.
52 https://www.pcwelt.de/ratgeber/Superscoring-Wie-wertvoll-sind-Sie-fuer-die-Gesellschaft-106
33488.html.