140
Biomedical Signal and Image Processing
that data using a Gaussian distribution, we attempt to find the best parameters
for some other distributions, because in order to emulate real problems, we must
assume that no knowledge of the true distribution is available.
With MLE giving us the most likely PDFs for classes, one can then use methods
such as Bayesian classifiers to perform classification. However, as previously mentioned, for most sophisticated problems, one may need to use more sophisticated
classifiers such neural networks, as discussed in the following.
7.7 NEURAL NETWORKS
Even though maximum likelihood and other statistical methods prove to be effective
techniques in rather simple problems, they may not provide the best solutions for
more complex problems. In addition, the need to make an assumption on the type of
the distribution function producing data, as explained earlier, is another disadvantage
of such methods.
In practice, methods such as neural networks are preferred in more sophisticated
problems. Neural networks are computational methods directly inspired by the formation and function of biological neural structures. Just like the biological neural
structures, artificial neural networks are simply a formation of artificial neurons that
learn patterns directly from examples. These methods are distribution free, which
means no assumptions are made on the type of the data distribution. In addition,
due to the nonlinear nature of the method, neural networks are capable of solving complex nonlinear classification problems that cannot be addressed by simple
statistical methods. Neural networks are composed of a number of neurons that are
connected through some “weights.” Since the structure and the function of each
neuron is known, in order to train a neural network, one needs to use the given training examples to find the best values of the weights. Once the weights are found, the
neural network is uniquely defined and can be used for classification and modeling.
Despite all capabilities listed for neural networks, the reader needs to be warned
not to use neural networks for certain types of applications. For example, neural networks require many training examples to learn a pattern, and if, for the problem to
be addressed, sufficient number of examples are not available, neural networks may
not be the best solutions. This is due to the fact that neural networks can easily fit
any training data and if the training dataset is too small to be a good representative
of the problem, neural networks are simply overfitting the training examples without
learning the true patterns.
There are many families of neural networks, but, here, we focus on the most
popular one, which is the family of multilayer sigmoid neural networks. These structures are supervised classification methods that can theoretically model any complex
system to any desired accuracy. We start describing these structures with a simple
single-neuron structure known as a perceptron.
7.7.1 PERCEPTRON
Perceptrons are nothing but simple emulation of the biological neurons. These
classifiers, as the simplest forms of neural networks, have a simple but effective
learning procedure. More specifically, given that the classes can be separated by
Biomedical Signal and Image Processing
that data using a Gaussian distribution, we attempt to find the best parameters
for some other distributions, because in order to emulate real problems, we must
assume that no knowledge of the true distribution is available.
With MLE giving us the most likely PDFs for classes, one can then use methods
such as Bayesian classifiers to perform classification. However, as previously mentioned, for most sophisticated problems, one may need to use more sophisticated
classifiers such neural networks, as discussed in the following.
7.7 NEURAL NETWORKS
Even though maximum likelihood and other statistical methods prove to be effective
techniques in rather simple problems, they may not provide the best solutions for
more complex problems. In addition, the need to make an assumption on the type of
the distribution function producing data, as explained earlier, is another disadvantage
of such methods.
In practice, methods such as neural networks are preferred in more sophisticated
problems. Neural networks are computational methods directly inspired by the formation and function of biological neural structures. Just like the biological neural
structures, artificial neural networks are simply a formation of artificial neurons that
learn patterns directly from examples. These methods are distribution free, which
means no assumptions are made on the type of the data distribution. In addition,
due to the nonlinear nature of the method, neural networks are capable of solving complex nonlinear classification problems that cannot be addressed by simple
statistical methods. Neural networks are composed of a number of neurons that are
connected through some “weights.” Since the structure and the function of each
neuron is known, in order to train a neural network, one needs to use the given training examples to find the best values of the weights. Once the weights are found, the
neural network is uniquely defined and can be used for classification and modeling.
Despite all capabilities listed for neural networks, the reader needs to be warned
not to use neural networks for certain types of applications. For example, neural networks require many training examples to learn a pattern, and if, for the problem to
be addressed, sufficient number of examples are not available, neural networks may
not be the best solutions. This is due to the fact that neural networks can easily fit
any training data and if the training dataset is too small to be a good representative
of the problem, neural networks are simply overfitting the training examples without
learning the true patterns.
There are many families of neural networks, but, here, we focus on the most
popular one, which is the family of multilayer sigmoid neural networks. These structures are supervised classification methods that can theoretically model any complex
system to any desired accuracy. We start describing these structures with a simple
single-neuron structure known as a perceptron.
7.7.1 PERCEPTRON
Perceptrons are nothing but simple emulation of the biological neurons. These
classifiers, as the simplest forms of neural networks, have a simple but effective
learning procedure. More specifically, given that the classes can be separated by