316
C.H. Reick . A. Grünewald . B. Page
We start the discussion by distinguishing several causes for predictive
performance failures of Neural Networks:
Unpredictability. The data may not represent a phenomenon governed by a
common rule. It is obvious that in such a case any prediction method will fail.
Despite its triviality, this point is mentioned here, because Neural Networks are
often tentatively applied to phenomena whose predictability is questionable, like
in the case of stock returns.
Poor data. It may happen that in principle the considered phenomenon is
predictable, but the data used for training contain not enough information. Several
types of information deficiencies can be distinguished: First, there may be simply
not enough data available. Second, the data may be too noisy. Third, the data set
may be incomplete, in the sense that certain aspects of the phenomenon are not
represented by the data. And finaIly, the data may contain the wrong information.
This can happen if the phenomenon to be predicted is recently governed by a
different rule as before (instationarity). If the data used for training contain no or
only partial information on the present rule, the Neural Network performs the
predictions according to the past rule, and consequently fails.
Overadaptation. Training a Neural Network means to change its parameters
iteratively until it reproduces a given set of input-output relations with sufficient
accuracy. This accuracy is measured by the error between the intended and the
actual output. Except for the initial training phase, where strong fluctuations may
be observed, this error usually decreases monotonically during training. But when
the error is monitored for a different data set, that is not used for training, one
often observes that beyond a certain point the error increases. This indicates that a
good adaptation and a good generalization are conflicting aims. So, when
employing Neural Networks for predictive purposes, one has to take care that the
training process is terminated before an overadaptation occurs. This problem,
especially how to detect and prevent an overadaptation, will be discussed in more
detail in section III.
Underadaptation. As already mentioned, after an initial phase typically not
only the error for the training data decreases, but also the error for independent
data. So, terminating the training too early may lead to an underadaptation and a
reduced ability to distinguish different system states. This problem is easily
circumvented by sufficiently long training periods. More fundamental is the
problem, that the training process may stick to a local minimum in the error
landscape so that, although the Neural Network may in principle be capable of a
good adaptation, the optimal parameters are not found during training. This
problem is weIl known from numerical mathematics and no general solution exists
(see e.g. Press et al. (1986».
Unsuited network structure. It is a general experience that when changing the
various structural elements of a Neural Network, like the number of neurons, the
topology of their connections or the type of activation and output functions, its
predictive performance changes. Unfortunately there are no general rules wh at
C.H. Reick . A. Grünewald . B. Page
We start the discussion by distinguishing several causes for predictive
performance failures of Neural Networks:
Unpredictability. The data may not represent a phenomenon governed by a
common rule. It is obvious that in such a case any prediction method will fail.
Despite its triviality, this point is mentioned here, because Neural Networks are
often tentatively applied to phenomena whose predictability is questionable, like
in the case of stock returns.
Poor data. It may happen that in principle the considered phenomenon is
predictable, but the data used for training contain not enough information. Several
types of information deficiencies can be distinguished: First, there may be simply
not enough data available. Second, the data may be too noisy. Third, the data set
may be incomplete, in the sense that certain aspects of the phenomenon are not
represented by the data. And finaIly, the data may contain the wrong information.
This can happen if the phenomenon to be predicted is recently governed by a
different rule as before (instationarity). If the data used for training contain no or
only partial information on the present rule, the Neural Network performs the
predictions according to the past rule, and consequently fails.
Overadaptation. Training a Neural Network means to change its parameters
iteratively until it reproduces a given set of input-output relations with sufficient
accuracy. This accuracy is measured by the error between the intended and the
actual output. Except for the initial training phase, where strong fluctuations may
be observed, this error usually decreases monotonically during training. But when
the error is monitored for a different data set, that is not used for training, one
often observes that beyond a certain point the error increases. This indicates that a
good adaptation and a good generalization are conflicting aims. So, when
employing Neural Networks for predictive purposes, one has to take care that the
training process is terminated before an overadaptation occurs. This problem,
especially how to detect and prevent an overadaptation, will be discussed in more
detail in section III.
Underadaptation. As already mentioned, after an initial phase typically not
only the error for the training data decreases, but also the error for independent
data. So, terminating the training too early may lead to an underadaptation and a
reduced ability to distinguish different system states. This problem is easily
circumvented by sufficiently long training periods. More fundamental is the
problem, that the training process may stick to a local minimum in the error
landscape so that, although the Neural Network may in principle be capable of a
good adaptation, the optimal parameters are not found during training. This
problem is weIl known from numerical mathematics and no general solution exists
(see e.g. Press et al. (1986».
Unsuited network structure. It is a general experience that when changing the
various structural elements of a Neural Network, like the number of neurons, the
topology of their connections or the type of activation and output functions, its
predictive performance changes. Unfortunately there are no general rules wh at