The input layer is the word vector matrix obtained in the previous step. Different
from the common input forms of images, each line in the word vector matrix represents
a word. In order to ensure that the region slipped by convolution kernel is always a
complete word, the convolution kernel is required to only slip vertically in height. Its
width is guaranteed to be consistent with the dimension of the word vector. In this
paper, we use convolution kernels of various window sizes to perform convolution
operations and obtain vector representations of different sizes. Next, the operation of
the max-pooling layer is processed; that is, the location of the maximum value in the
upper results is selected. After the max-pooling layer is processed, a scalar is obtained.
Then, the scalars processed from the same size convolution kernel are joined together
to obtain the feature vectors under the same window size. The last layer is a fully
connected layer with dropout. Dropout is mainly designed to prevent over-fitting and
improve the generalization ability of the model. The final complete feature vectors are
the representation of short text after processing by CNN and also serve as the input of
clustering algorithm.
3 Clustering Ensemble of Short Text
Clustering is to discover the potential relationships between data structures and obtain
more useful information. However, there is no single clustering algorithm that is able to
deal with all types of data structures. Each clustering algorithm has its special advantages as well as weakness. Given the same data set, different clustering algorithms and
even different parameters of the same algorithm will lead to different clustering results,
but it is difficult to directly select any clustering results without prior knowledge.
Clustering ensemble algorithm combines various clustering algorithms to obtain a more
accurate clustering result. Clustering ensemble has shown advantages in finding bizarre
clusters and dealing with noise [10]. In clustering ensemble, each clustering as input is
called base clustering, and the final result is called consensus clustering.
3.1 Selecting Base Clustering
In this paper, several different types of classical clustering algorithms are selected as the
base clustering. They are K-means algorithm based on partition, AGNES algorithm
which is a typically hierarchical clustering algorithm, and DBSCAN algorithm which is
based on density.
K-means algorithm is a traditional partition-based clustering algorithm. The main
idea of the algorithm is roughly as follows: K samples are randomly selected from the
sample set as clustering centers, and the distance between all samples and the K clustering centers is calculated. For each sample, it is divided into the clustering centers
nearest to the center, and the new clustering centers of each cluster are calculated for
the new clustering.
AGENS is a hierarchical clustering algorithm based on bottom-up clustering
strategy. It considers each sample in the data set as an initial clustering and then
combines the two nearest clusters at each step of the algorithm. The process repeats
until the number of preset clusters is reached.
624
H. Wan et al.
from the common input forms of images, each line in the word vector matrix represents
a word. In order to ensure that the region slipped by convolution kernel is always a
complete word, the convolution kernel is required to only slip vertically in height. Its
width is guaranteed to be consistent with the dimension of the word vector. In this
paper, we use convolution kernels of various window sizes to perform convolution
operations and obtain vector representations of different sizes. Next, the operation of
the max-pooling layer is processed; that is, the location of the maximum value in the
upper results is selected. After the max-pooling layer is processed, a scalar is obtained.
Then, the scalars processed from the same size convolution kernel are joined together
to obtain the feature vectors under the same window size. The last layer is a fully
connected layer with dropout. Dropout is mainly designed to prevent over-fitting and
improve the generalization ability of the model. The final complete feature vectors are
the representation of short text after processing by CNN and also serve as the input of
clustering algorithm.
3 Clustering Ensemble of Short Text
Clustering is to discover the potential relationships between data structures and obtain
more useful information. However, there is no single clustering algorithm that is able to
deal with all types of data structures. Each clustering algorithm has its special advantages as well as weakness. Given the same data set, different clustering algorithms and
even different parameters of the same algorithm will lead to different clustering results,
but it is difficult to directly select any clustering results without prior knowledge.
Clustering ensemble algorithm combines various clustering algorithms to obtain a more
accurate clustering result. Clustering ensemble has shown advantages in finding bizarre
clusters and dealing with noise [10]. In clustering ensemble, each clustering as input is
called base clustering, and the final result is called consensus clustering.
3.1 Selecting Base Clustering
In this paper, several different types of classical clustering algorithms are selected as the
base clustering. They are K-means algorithm based on partition, AGNES algorithm
which is a typically hierarchical clustering algorithm, and DBSCAN algorithm which is
based on density.
K-means algorithm is a traditional partition-based clustering algorithm. The main
idea of the algorithm is roughly as follows: K samples are randomly selected from the
sample set as clustering centers, and the distance between all samples and the K clustering centers is calculated. For each sample, it is divided into the clustering centers
nearest to the center, and the new clustering centers of each cluster are calculated for
the new clustering.
AGENS is a hierarchical clustering algorithm based on bottom-up clustering
strategy. It considers each sample in the data set as an initial clustering and then
combines the two nearest clusters at each step of the algorithm. The process repeats
until the number of preset clusters is reached.
624
H. Wan et al.