272
Phosphorous (mg PO. -P I L)
Nitrogen (mg N0 3 -N I L)
Secchi Disc Depth (m) -
Water Temperature (deg C) -
input layer
Chlorophyll a (ug/L) -
Phosphorous (mg PO. -P I L)
Nitrogen (mg N0 3 -N I L)
-
Secchi Disc Depth (m) -
Water Temperature (deg C) -
input layer
H. Wilson . F. Recknagel
a
-
Chlorophyll a (ugIL)
hidden layer
output layer
-
hidden layer
output layer
b
Chlorophyll a (ugIL)
next month
Figure 14.2. a Generic ANN design for same day model predictions (inputs and
outputs are at the same time step). b Generic ANN design for 30 day ahead model
predictions (inputs lag the outputs by 30 days).
14.3.2
Model Approximation (Training)
The Stuttgart Neural Network Simulator version 4.1 software package
(http://www-ra.infarmatik.uni-tuebingen.de/SNNSI) was used far building and
training ANNs. It was elected to use the scaled conjugate gradient (SCG) training
method (Ms;;ller 1993) in preference to backpropagation because preliminary
experimentation showed it to be an order of magnitude faster whilst achieving
similar generalisation performance on independent test data. Also, unlike
backpropagation, SCG requires no tuning of meta-parameters such as learning rate
or momentum to optimise performance, and is conditionally guaranteed to
converge on a minimum of the objective function (unlike backpropagation with
static learning rate which converges on a limit cyde around the minimum).
Preliminary experiments also demonstrated that training with raw data led to
slow convergence and frequent entrapment at bad local optima. However, when
the data was conditioned by scaling such that it had a mean of 0 and a standard
deviation of 1, the learning performance was greatly improved. Output data was
scaled to be consistent with the sigmoidal transfer function of the output neuron
(i.e between 0 and 1). Prior to each training ron, the network weights were
randomly initialised to fall between -0.1 and 0.1, with the random number
generator being reseeded with the current time in each case.
In preliminary experimentation it was found that average learning time for the
ANN designs investigated fell from minutes/hours to less than 10 seconds on a
contemporary desktop CPU by implementation of the measures outlined above.
ANN training information is summarised in table 14.2.
Phosphorous (mg PO. -P I L)
Nitrogen (mg N0 3 -N I L)
Secchi Disc Depth (m) -
Water Temperature (deg C) -
input layer
Chlorophyll a (ug/L) -
Phosphorous (mg PO. -P I L)
Nitrogen (mg N0 3 -N I L)
-
Secchi Disc Depth (m) -
Water Temperature (deg C) -
input layer
H. Wilson . F. Recknagel
a
-
Chlorophyll a (ugIL)
hidden layer
output layer
-
hidden layer
output layer
b
Chlorophyll a (ugIL)
next month
Figure 14.2. a Generic ANN design for same day model predictions (inputs and
outputs are at the same time step). b Generic ANN design for 30 day ahead model
predictions (inputs lag the outputs by 30 days).
14.3.2
Model Approximation (Training)
The Stuttgart Neural Network Simulator version 4.1 software package
(http://www-ra.infarmatik.uni-tuebingen.de/SNNSI) was used far building and
training ANNs. It was elected to use the scaled conjugate gradient (SCG) training
method (Ms;;ller 1993) in preference to backpropagation because preliminary
experimentation showed it to be an order of magnitude faster whilst achieving
similar generalisation performance on independent test data. Also, unlike
backpropagation, SCG requires no tuning of meta-parameters such as learning rate
or momentum to optimise performance, and is conditionally guaranteed to
converge on a minimum of the objective function (unlike backpropagation with
static learning rate which converges on a limit cyde around the minimum).
Preliminary experiments also demonstrated that training with raw data led to
slow convergence and frequent entrapment at bad local optima. However, when
the data was conditioned by scaling such that it had a mean of 0 and a standard
deviation of 1, the learning performance was greatly improved. Output data was
scaled to be consistent with the sigmoidal transfer function of the output neuron
(i.e between 0 and 1). Prior to each training ron, the network weights were
randomly initialised to fall between -0.1 and 0.1, with the random number
generator being reseeded with the current time in each case.
In preliminary experimentation it was found that average learning time for the
ANN designs investigated fell from minutes/hours to less than 10 seconds on a
contemporary desktop CPU by implementation of the measures outlined above.
ANN training information is summarised in table 14.2.