d. Now, assume we know that data points are created by Gaussian random generators as follows: the first 100 data points centered at (1, 1), the second 100
data points centered at (1, 0), the third 100 data points at (0, 1), and the last
100 points at (0, 0). With that in mind, compare the results of the last three
parts and comment on the sensitivity of the K-means clustering technique to
the initialization and the convergence rate to the final cluster centers.
7.2 W e are to use a perceptron to model logical “AND” function (gate) with two inputs.
The output of an AND gate is 1 only if both inputs are 1. Form the training set and
manually train the perceptron. Assume the learning rate of 0.5 and initialize the
weights as follows: w 1 = 1, w 1 = −1, and w 1 = 1. Use binary inputs and binary outputs.
7.3 Repeat Problem 7.2 to model the logical “NOR” function. The output of a NOR
gate is 1 only if both inputs are 0. Form the training set and manually train the
perceptron. Assume the learning rate of 0.5 and initialize the weights as follows:
w1 = 1, w1 = −1, and w1 = 1. Use binary inputs and binary output.
7.4 P rove that perceptron cannot be trained to model “XOR” gate. The output of a
XOR gate is 1 only if both inputs have different values, i.e., if one input is 0, the
other one must be 1 to have 1 as the output. Inability of perceptron is a historical
observation that changed the direction of the field of neural networks.
7.5 Two conditional probability functions P(x|ω 1 ) and P(x|ω 2 ) are given as follows:
1
⎡ (x −1)
2
⎤
p x
1 ( ) =
exp ⎢ −
⎥
(7.34)
2p
2
⎣
⎦
and
p x
2 ( ) = A exp ( − x )
(7.35)
a. Find the value of A.
b. Assuming that P(ω 1 ) = P(ω 2 ), find the border (in terms of x) for the Bayesian
decision.
c. I f the observation x = 0.5 is made, according the Bayesian classifier designed
in part “b,” which class does this sample belong to?
7.6 A set of observation, generated by an unknown probability function, is given as
follows:
8.5, 7.2, 12.6, 11.1, 8.4, 9.4, 10.3, 6.5.
Consider the following probability functions:
1
⎡ (x −1)
2
⎤
p x
1 ( ) =
exp ⎢ −
⎥
(7.36)
2p
2
⎣
⎦
and
1
⎡ (x −10)
2
⎤
p x
1 ( ) =
exp ⎢ −
⎥
(7.37)
2p
2
⎣
⎦
151
Clustering and Classification
data points centered at (1, 0), the third 100 data points at (0, 1), and the last
100 points at (0, 0). With that in mind, compare the results of the last three
parts and comment on the sensitivity of the K-means clustering technique to
the initialization and the convergence rate to the final cluster centers.
7.2 W e are to use a perceptron to model logical “AND” function (gate) with two inputs.
The output of an AND gate is 1 only if both inputs are 1. Form the training set and
manually train the perceptron. Assume the learning rate of 0.5 and initialize the
weights as follows: w 1 = 1, w 1 = −1, and w 1 = 1. Use binary inputs and binary outputs.
7.3 Repeat Problem 7.2 to model the logical “NOR” function. The output of a NOR
gate is 1 only if both inputs are 0. Form the training set and manually train the
perceptron. Assume the learning rate of 0.5 and initialize the weights as follows:
w1 = 1, w1 = −1, and w1 = 1. Use binary inputs and binary output.
7.4 P rove that perceptron cannot be trained to model “XOR” gate. The output of a
XOR gate is 1 only if both inputs have different values, i.e., if one input is 0, the
other one must be 1 to have 1 as the output. Inability of perceptron is a historical
observation that changed the direction of the field of neural networks.
7.5 Two conditional probability functions P(x|ω 1 ) and P(x|ω 2 ) are given as follows:
1
⎡ (x −1)
2
⎤
p x
1 ( ) =
exp ⎢ −
⎥
(7.34)
2p
2
⎣
⎦
and
p x
2 ( ) = A exp ( − x )
(7.35)
a. Find the value of A.
b. Assuming that P(ω 1 ) = P(ω 2 ), find the border (in terms of x) for the Bayesian
decision.
c. I f the observation x = 0.5 is made, according the Bayesian classifier designed
in part “b,” which class does this sample belong to?
7.6 A set of observation, generated by an unknown probability function, is given as
follows:
8.5, 7.2, 12.6, 11.1, 8.4, 9.4, 10.3, 6.5.
Consider the following probability functions:
1
⎡ (x −1)
2
⎤
p x
1 ( ) =
exp ⎢ −
⎥
(7.36)
2p
2
⎣
⎦
and
1
⎡ (x −10)
2
⎤
p x
1 ( ) =
exp ⎢ −
⎥
(7.37)
2p
2
⎣
⎦
151
Clustering and Classification