74
4 Google as God?
4.8 Limitations of the “Crystal Ball”
One might think that these three kinds of errors could be overcome if we just had
enough data. But is this really true? There are a number of fundamental scientific
factors that will impair the predictive power of the crystal ball. For example, the
problem, which is known as “Laplace’s Demon”, posits that all future developments
are determined by the world’s history. As a result, our ability to make predictions
is fundamentally constrained by our inability to measure all historical information
needed to predict the future
41 (assuming that the world would change according to
deterministic rules at all,
42 which is questionable).
In general, the parameters of a computer model representing certain aspects of
our world can only be determined with a certain accuracy. But even a small variation
of the assumed model parameters within their respective “confidence intervals” may
fundamentally alter the model predictions. In complex anthropogenic systems, such
“parameter sensitivity” is often expected to be large. While the confidence intervals
may be narrowed down, if enough data are available, having too much data can be a
problem, too: it may reduce the quality of predictions due to “over-fitting”, “spurious
correlations” or “herding effects”.
Furthermore, many complex dynamical systems show phenomena such as “turbulence” or “(deterministic) chaos”, for which even the slightest difference may fundamentally change the outcome sooner or later. This well-known property is sometimes
called the “butterfly effect”. It imposes a time limit beyond which no useful forecast
can be made. The particular physics of weather phenomena is the reason why meteorologists cannot forecast the weather reasonably well for more than a few days,
43
and even a million times more data could not fundamentally change this.
In social systems there is the additional problem of ambiguity: the same information may have several different meanings depending on the context, and the way we
interpret it may influence the future course of the system. Beyond this, since Kurt
Gödel (1906–1978) it is known that some questions are fundamentally undecidable
in that the correctness of certain statements can neither be proved nor disproved with
formal logic. Appendix 4.2 explains the above problems in more detail.
Thus, it is safe to say that Big Data is not the universal panacea it is often claimed
to be.
44 Attempts to predict the future will be mostly limited to probabilistic and
short-term forecasts. This particularly applies to unstable systems. It is, therefore,
41 For example, we are influenced by cultural inventions, ideas and social norms which are sometimes
thousands of years old.
42 Assuming determinism is rather questionable, given that there are probably many random
influences on the world’s course, as we know from quantum mechanics, for example.
43 These limitations in predictive power are not merely a matter of having insufficient data or not
enough computer power. The physical nature of the underlying processes fundamentally limits the
precision of forecasts.
44 To convince me otherwise, in an analogy to the “Turing test” which checks whether a computer
can communicate in a manner which is undistinguishable from a human, a computer system would
have to find all the fundamental laws of physics discovered by scientists so far by mining the
experimental data accumulated in the past centuries.
4 Google as God?
4.8 Limitations of the “Crystal Ball”
One might think that these three kinds of errors could be overcome if we just had
enough data. But is this really true? There are a number of fundamental scientific
factors that will impair the predictive power of the crystal ball. For example, the
problem, which is known as “Laplace’s Demon”, posits that all future developments
are determined by the world’s history. As a result, our ability to make predictions
is fundamentally constrained by our inability to measure all historical information
needed to predict the future
41 (assuming that the world would change according to
deterministic rules at all,
42 which is questionable).
In general, the parameters of a computer model representing certain aspects of
our world can only be determined with a certain accuracy. But even a small variation
of the assumed model parameters within their respective “confidence intervals” may
fundamentally alter the model predictions. In complex anthropogenic systems, such
“parameter sensitivity” is often expected to be large. While the confidence intervals
may be narrowed down, if enough data are available, having too much data can be a
problem, too: it may reduce the quality of predictions due to “over-fitting”, “spurious
correlations” or “herding effects”.
Furthermore, many complex dynamical systems show phenomena such as “turbulence” or “(deterministic) chaos”, for which even the slightest difference may fundamentally change the outcome sooner or later. This well-known property is sometimes
called the “butterfly effect”. It imposes a time limit beyond which no useful forecast
can be made. The particular physics of weather phenomena is the reason why meteorologists cannot forecast the weather reasonably well for more than a few days,
43
and even a million times more data could not fundamentally change this.
In social systems there is the additional problem of ambiguity: the same information may have several different meanings depending on the context, and the way we
interpret it may influence the future course of the system. Beyond this, since Kurt
Gödel (1906–1978) it is known that some questions are fundamentally undecidable
in that the correctness of certain statements can neither be proved nor disproved with
formal logic. Appendix 4.2 explains the above problems in more detail.
Thus, it is safe to say that Big Data is not the universal panacea it is often claimed
to be.
44 Attempts to predict the future will be mostly limited to probabilistic and
short-term forecasts. This particularly applies to unstable systems. It is, therefore,
41 For example, we are influenced by cultural inventions, ideas and social norms which are sometimes
thousands of years old.
42 Assuming determinism is rather questionable, given that there are probably many random
influences on the world’s course, as we know from quantum mechanics, for example.
43 These limitations in predictive power are not merely a matter of having insufficient data or not
enough computer power. The physical nature of the underlying processes fundamentally limits the
precision of forecasts.
44 To convince me otherwise, in an analogy to the “Turing test” which checks whether a computer
can communicate in a manner which is undistinguishable from a human, a computer system would
have to find all the fundamental laws of physics discovered by scientists so far by mining the
experimental data accumulated in the past centuries.