141
Clustering and Classification
straight lines, perceptrons can always provide the correct classifiers. Figure 7.5
shows the structure of a simple perceptron. Just as in a biological neuron that
receives some inputs from the neighboring neurons and produces an output, a perceptron has a number of inputs and one output. As can be seen, an extra input,
called bias, is added to the regular inputs. As discussed in more detail later in this
chapter, even though the value of this input is constant, the insertion of the bias
input helps improve the classification performance.
As can be seen in Figure 7.5, there is a weight associated with the connection
between any of the inputs x i ’s and the output of the neuron. The strength of each connection is described by a weight w i , and the weight of the bias input is denoted by b.
The total effect of the inputs on the perceptron is given as the weighted summation
of the inputs:
y 0 = +
b ∑ x i i
w
(7.15)
i
Then, the output of the perceptron is calculated based on y 0 as follows:
⎧1
if y 0 > T
⎪ ⎪
y = ⎨ 0
if − <
T y 0 < T
(7.16)
⎪
⎪ ⎩ −1 if y 0 < −T
where T is a threshold value that is often set by the user.
The thresholding process shown in Equation 7.14 is exactly what is happening in biological neurons, i.e., if the total excitation of the neighboring neurons is
more than a threshold value, the neuron fires. The choice of the threshold value
T is rather arbitrary, but when this value is chosen, it is fixed throughout training
and testing processes. It has to be mentioned that perceptrons can be designed to
accept or produce binary (0 or 1), bipolar (1 and −1), or even real values. The earlier
formulation, i.e., bipolar, can be easily modified to create binary and real-valued
models too.
As briefly mentioned earlier, besides the regular inputs to a perceptron, which
are the features of the samples, it is often necessary to add another input, which
is always set to a constant such as 1. This input, which is called bias, is needed to
produce better separation among the classes of patterns. Knowing that bias is always
set to 1, it is evident that no new information is added to the network by adding the
bias, but a new term is added to the summation terms in y 0 . In addition, the weight
connecting this constant input to the perceptron needs to be updated as any other
weights in the system.
Next, we discuss the training of a perceptron. Note that the main goal of training
a perceptron is to teach the perceptron how to classify each input pattern and determine the particular class the pattern belongs to. In other words, training is nothing
but to adjust the weights of the perceptron to make it produce 1 if the sample belongs
Clustering and Classification
straight lines, perceptrons can always provide the correct classifiers. Figure 7.5
shows the structure of a simple perceptron. Just as in a biological neuron that
receives some inputs from the neighboring neurons and produces an output, a perceptron has a number of inputs and one output. As can be seen, an extra input,
called bias, is added to the regular inputs. As discussed in more detail later in this
chapter, even though the value of this input is constant, the insertion of the bias
input helps improve the classification performance.
As can be seen in Figure 7.5, there is a weight associated with the connection
between any of the inputs x i ’s and the output of the neuron. The strength of each connection is described by a weight w i , and the weight of the bias input is denoted by b.
The total effect of the inputs on the perceptron is given as the weighted summation
of the inputs:
y 0 = +
b ∑ x i i
w
(7.15)
i
Then, the output of the perceptron is calculated based on y 0 as follows:
⎧1
if y 0 > T
⎪ ⎪
y = ⎨ 0
if − <
T y 0 < T
(7.16)
⎪
⎪ ⎩ −1 if y 0 < −T
where T is a threshold value that is often set by the user.
The thresholding process shown in Equation 7.14 is exactly what is happening in biological neurons, i.e., if the total excitation of the neighboring neurons is
more than a threshold value, the neuron fires. The choice of the threshold value
T is rather arbitrary, but when this value is chosen, it is fixed throughout training
and testing processes. It has to be mentioned that perceptrons can be designed to
accept or produce binary (0 or 1), bipolar (1 and −1), or even real values. The earlier
formulation, i.e., bipolar, can be easily modified to create binary and real-valued
models too.
As briefly mentioned earlier, besides the regular inputs to a perceptron, which
are the features of the samples, it is often necessary to add another input, which
is always set to a constant such as 1. This input, which is called bias, is needed to
produce better separation among the classes of patterns. Knowing that bias is always
set to 1, it is evident that no new information is added to the network by adding the
bias, but a new term is added to the summation terms in y 0 . In addition, the weight
connecting this constant input to the perceptron needs to be updated as any other
weights in the system.
Next, we discuss the training of a perceptron. Note that the main goal of training
a perceptron is to teach the perceptron how to classify each input pattern and determine the particular class the pattern belongs to. In other words, training is nothing
but to adjust the weights of the perceptron to make it produce 1 if the sample belongs