390
G.M. Foody
20.3.1.3
Probabilistic Neural Networks (PNN)
The PNN is used only for classification problems and is typically very much
larger than an MLP or RBF network designed to undertake the same task. With
the PNN network, estimates of the probability density functions (p.d.f.) of the
possible classes are derived and used to select the most probable class of
membership for each case. Rather than assurne a certain form for the p.d.f. (e.g. a
normal distribution) a kernel-based approximation is commonly used (Bishop
1995). With this, simple functions are located at each available case and added
together to derive an estimate of the overall p.d.f. This kernel-based approach to
p.d.f approximation has similarities to aspects of the RBF networks. Indeed, radial
units are contained in the hidden layer of a PNN. In total, the PNN contains at
least three layers with the input and output layers performing similar functions to
the other network types. The radial units of the PNN essentially copy the training
data and hence the number of hidden radial units is equal to the size of the training
set. PNN networks are consequently typically very much larger than MLP and
RBF networks designed for the same problem. Each radial unit models a Gaussian
function centred on the relevant training case and is connected to the unit in the
output layer associated with the actual class of membership for that training case.
Thus each output unit is connected to only the radial units associated with the
same class with a zero connection to all other radial units. In the output units, the
sum of all the responses of the radial units bel on ging to the relevant class is
derived and these are proportional to the kernel based estimates of the p.d.f.'s of
the defined classes. These outputs may be normalised to sum to unity to provide a
probabilistic output and each case of previously unknown membership may be
allocated to the class with which it has the highest probability of membership.
This type of network can be rapid to train but, because of its size, slow to apply to
large data sets. It has a major advantage over the other network types of being
able to readily accommodate prior information into the analysis, which is often
advantageous in remote sensing applications (Strahler 1980; Mather 1999) and
can be used to increase classification accuracy (Foody 2001).
20.3.1.4
Generalised Regression Neural Network (GRNN)
The GRNN is a regression (Bayesian) network that has some similarities with the
PNN. The GRNN, however, has four layers and is used only for regression
problems. First, is the input layer as normal. Second, a layer of radial units that are
used to provide a representation of the centres of clusters identified from the
training sample. The radial layer is typically large, but usually smaller than the
size the training sampie, and is trained using a clustering algorithm. The third
layer in the network is a layer of regression units. Two types of unit are
encountered in this layer, one calculates the desired regression outputs while the
other calculates the probability density. In total there is always one more unit in
G.M. Foody
20.3.1.3
Probabilistic Neural Networks (PNN)
The PNN is used only for classification problems and is typically very much
larger than an MLP or RBF network designed to undertake the same task. With
the PNN network, estimates of the probability density functions (p.d.f.) of the
possible classes are derived and used to select the most probable class of
membership for each case. Rather than assurne a certain form for the p.d.f. (e.g. a
normal distribution) a kernel-based approximation is commonly used (Bishop
1995). With this, simple functions are located at each available case and added
together to derive an estimate of the overall p.d.f. This kernel-based approach to
p.d.f approximation has similarities to aspects of the RBF networks. Indeed, radial
units are contained in the hidden layer of a PNN. In total, the PNN contains at
least three layers with the input and output layers performing similar functions to
the other network types. The radial units of the PNN essentially copy the training
data and hence the number of hidden radial units is equal to the size of the training
set. PNN networks are consequently typically very much larger than MLP and
RBF networks designed for the same problem. Each radial unit models a Gaussian
function centred on the relevant training case and is connected to the unit in the
output layer associated with the actual class of membership for that training case.
Thus each output unit is connected to only the radial units associated with the
same class with a zero connection to all other radial units. In the output units, the
sum of all the responses of the radial units bel on ging to the relevant class is
derived and these are proportional to the kernel based estimates of the p.d.f.'s of
the defined classes. These outputs may be normalised to sum to unity to provide a
probabilistic output and each case of previously unknown membership may be
allocated to the class with which it has the highest probability of membership.
This type of network can be rapid to train but, because of its size, slow to apply to
large data sets. It has a major advantage over the other network types of being
able to readily accommodate prior information into the analysis, which is often
advantageous in remote sensing applications (Strahler 1980; Mather 1999) and
can be used to increase classification accuracy (Foody 2001).
20.3.1.4
Generalised Regression Neural Network (GRNN)
The GRNN is a regression (Bayesian) network that has some similarities with the
PNN. The GRNN, however, has four layers and is used only for regression
problems. First, is the input layer as normal. Second, a layer of radial units that are
used to provide a representation of the centres of clusters identified from the
training sample. The radial layer is typically large, but usually smaller than the
size the training sampie, and is trained using a clustering algorithm. The third
layer in the network is a layer of regression units. Two types of unit are
encountered in this layer, one calculates the desired regression outputs while the
other calculates the probability density. In total there is always one more unit in