120
V. Tra and J.-M. Kim
It is common knowledge that the best solution is the feature-set that represents
categories in the way that minimizes the intracategory compactness and maximizes
the intercategory separability between them. From (11.4) and (11.5), it is clear that
the two above conditions is proportional to the small value of R B F between (F mat ),
and the large value of R B F within (F mat ). The fitness function is ultimately defined as
follows:
Obj (F mat ) = 1 − R B F within (F mat ) + R B F between (F mat ).
(11.6)
11.2.3 Feature Evaluation Metric Using k-NN-Based
Classifier
11.2.3.1 k-NN-Based Classifier
With the subtle architecture and the ability to generalize classification problems
quickly with a limited dataset, the k-nearest neighbors (k-NN) algorithm has been
widely used as a flexible and straightforward classifier. As a type of a non-parametric
method, the k-NN uses a voting scheme to classify samples. More specifically, to
determine what category a specific sample belongs to, the k-NN utilizes the information of voting papers of that sample’s nearest neighbors. This means that there is
no training phase in the k-NN algorithm and to solve a classification task, training
samples are stored in memory and are used as reference points to determine the
category of a new sample. Operating with such the mechanism, the input data needs
to be pre-processed in a way that reduces the dimension of the input feature vector.
Without this procedure, the k-NN classifier will certainly suffer from severe computational complexity. For this reason, it is extremely important to adopt a featureselection scheme to select discriminatory features among potential ones and reduce
the dimension of the input vector as well.
To design the k-NN classifier, researchers need to pick up and tune carefully
two vital parameters (i.e., the parameter k that defines the number of neighbors of
an objective and the type of distance metric). While the value of k can be chosen
randomly or through a cross-validation process, the distance metric is chosen among
popular ones. In this study, the Euclidian distance is used as the metric due to its
effectiveness and simplicity.
V. Tra and J.-M. Kim
It is common knowledge that the best solution is the feature-set that represents
categories in the way that minimizes the intracategory compactness and maximizes
the intercategory separability between them. From (11.4) and (11.5), it is clear that
the two above conditions is proportional to the small value of R B F between (F mat ),
and the large value of R B F within (F mat ). The fitness function is ultimately defined as
follows:
Obj (F mat ) = 1 − R B F within (F mat ) + R B F between (F mat ).
(11.6)
11.2.3 Feature Evaluation Metric Using k-NN-Based
Classifier
11.2.3.1 k-NN-Based Classifier
With the subtle architecture and the ability to generalize classification problems
quickly with a limited dataset, the k-nearest neighbors (k-NN) algorithm has been
widely used as a flexible and straightforward classifier. As a type of a non-parametric
method, the k-NN uses a voting scheme to classify samples. More specifically, to
determine what category a specific sample belongs to, the k-NN utilizes the information of voting papers of that sample’s nearest neighbors. This means that there is
no training phase in the k-NN algorithm and to solve a classification task, training
samples are stored in memory and are used as reference points to determine the
category of a new sample. Operating with such the mechanism, the input data needs
to be pre-processed in a way that reduces the dimension of the input feature vector.
Without this procedure, the k-NN classifier will certainly suffer from severe computational complexity. For this reason, it is extremely important to adopt a featureselection scheme to select discriminatory features among potential ones and reduce
the dimension of the input vector as well.
To design the k-NN classifier, researchers need to pick up and tune carefully
two vital parameters (i.e., the parameter k that defines the number of neighbors of
an objective and the type of distance metric). While the value of k can be chosen
randomly or through a cross-validation process, the distance metric is chosen among
popular ones. In this study, the Euclidian distance is used as the metric due to its
effectiveness and simplicity.