128
6 A Planetary-Scale Threat
government research funding and philosophical thought. In Apocalyptic AI ,
Robert Geraci offers the first serious account of this “cyber-theology“ and the
people who promote it.
Drawing on interviews with roboticists and AI researchers and with devotees
of the online game Second Life, among others, Geraci illuminates the ideas
of such advocates of Apocalyptic AI as Hans Moravec and Ray Kurzweil. He
reveals that the rhetoric of Apocalyptic AI is strikingly similar to that of the
apocalyptic traditions of Judaism and Christianity. In both systems, the believer
is trapped in a dualistic universe and expects a resolution in which he or she
will be translated to a transcendent new world and live forever in a glorified
new body. Equally important, Geraci shows how this worldview shapes our
culture. Apocalyptic AI has become a powerful force in modern culture. In this
superb volume, he shines a light on this belief system, revealing what it is and
how it is changing society.”
I also recommend to read the book review over there by “sqee” posted on
December 16, 2010. It summarizes the apocalyptic elements of the Judaic/Christian
theology, which some transhumanists are now trying to engineer, including:
“A. A belief that there will be an irreversible event on a massive scale
(global) after which nothing will ever be the same (in traditional apocalypses,
the apocalypse itself; in Apocalyptic AI ideology, an event known as “the
singularity“).
B. A belief that after the apocalypse/singularity, rewards will be granted to
followers/adherents/believers that completely transform the experience of life
as we know it [while the others are apparently doomed].”
Personally, I don’t believe in this vision. I rather consider the above as an “apocalyptic” worst-case scenario that may happen if we don’t manage to avert attempts to
steer human behaviors and minds, and submit humanity to a (digital) control system.
It is clear that such a system would not establish a “golden age of peace and prosperity”, but would be an “evil” totalitarian system that would challenge the future
of humanity altogether (as Elon Musk has warned). Even though some tech companies and visionaries seem to promote the underlying technological developments,
we should stay away from ethically problematic applications—particularly those
that endanger human dignity, human rights, and democratic governance systems.
It is high time to challenge the “technology-driven” approach. Technology should
serve humans, not the other way round. In fact, in the further chapters I will illustrate
that human-machine convergence
88 is not the only possible future scenario. I would
88 The Convergence of Humans and Machines, IEEE Future Directions (October 29, 2017) https://
cmte.ieee.org/futuredirections/2017/10/29/the-convergence-of-humans-and-machines/.
6 A Planetary-Scale Threat
government research funding and philosophical thought. In Apocalyptic AI ,
Robert Geraci offers the first serious account of this “cyber-theology“ and the
people who promote it.
Drawing on interviews with roboticists and AI researchers and with devotees
of the online game Second Life, among others, Geraci illuminates the ideas
of such advocates of Apocalyptic AI as Hans Moravec and Ray Kurzweil. He
reveals that the rhetoric of Apocalyptic AI is strikingly similar to that of the
apocalyptic traditions of Judaism and Christianity. In both systems, the believer
is trapped in a dualistic universe and expects a resolution in which he or she
will be translated to a transcendent new world and live forever in a glorified
new body. Equally important, Geraci shows how this worldview shapes our
culture. Apocalyptic AI has become a powerful force in modern culture. In this
superb volume, he shines a light on this belief system, revealing what it is and
how it is changing society.”
I also recommend to read the book review over there by “sqee” posted on
December 16, 2010. It summarizes the apocalyptic elements of the Judaic/Christian
theology, which some transhumanists are now trying to engineer, including:
“A. A belief that there will be an irreversible event on a massive scale
(global) after which nothing will ever be the same (in traditional apocalypses,
the apocalypse itself; in Apocalyptic AI ideology, an event known as “the
singularity“).
B. A belief that after the apocalypse/singularity, rewards will be granted to
followers/adherents/believers that completely transform the experience of life
as we know it [while the others are apparently doomed].”
Personally, I don’t believe in this vision. I rather consider the above as an “apocalyptic” worst-case scenario that may happen if we don’t manage to avert attempts to
steer human behaviors and minds, and submit humanity to a (digital) control system.
It is clear that such a system would not establish a “golden age of peace and prosperity”, but would be an “evil” totalitarian system that would challenge the future
of humanity altogether (as Elon Musk has warned). Even though some tech companies and visionaries seem to promote the underlying technological developments,
we should stay away from ethically problematic applications—particularly those
that endanger human dignity, human rights, and democratic governance systems.
It is high time to challenge the “technology-driven” approach. Technology should
serve humans, not the other way round. In fact, in the further chapters I will illustrate
that human-machine convergence
88 is not the only possible future scenario. I would
88 The Convergence of Humans and Machines, IEEE Future Directions (October 29, 2017) https://
cmte.ieee.org/futuredirections/2017/10/29/the-convergence-of-humans-and-machines/.