296
F. Firouzi et al.
5.5 Dimensionality Reduction
Computers cannot think in the way that human brains do, and developing neural
networks is an attempt to address this issue. An artificial neural network, first
developed in the 1950s, is a simulation of the neurons of the human brains in a
manner that the computer can learn things in the humankind.
Dimensionality reduction is the process of reducing the number of machine
learning features and the dimension of the feature set. The key idea behind
dimensionality reduction is to convert a large set of dependent and correlated
features (which have redundant information) to a small set of independent features.
This enables us to remove the redundant information in the dataset which in turn
improves the speed as well as the performance of machine learning algorithms.
One of the most well-known techniques for dimensionality reduction is principal
component analysis (PCA). In PCA, the directions with the largest variances are
considered as most “important” (i.e., the most principal) features. Therefore, to find
the most important features, one needs to find the directions of the data that have the
maximum amount of variance. PCA algorithm can be summarized in the following
steps (see Fig. 5.50):
• The first step is to perform standardization (i.e., subtracting the mean and
dividing by the standard deviation).
• The next step is to find the covariance matrix of the features.
# Principal Component
% Variance Explained
Fig. 5.50 An illustration of PCA
F. Firouzi et al.
5.5 Dimensionality Reduction
Computers cannot think in the way that human brains do, and developing neural
networks is an attempt to address this issue. An artificial neural network, first
developed in the 1950s, is a simulation of the neurons of the human brains in a
manner that the computer can learn things in the humankind.
Dimensionality reduction is the process of reducing the number of machine
learning features and the dimension of the feature set. The key idea behind
dimensionality reduction is to convert a large set of dependent and correlated
features (which have redundant information) to a small set of independent features.
This enables us to remove the redundant information in the dataset which in turn
improves the speed as well as the performance of machine learning algorithms.
One of the most well-known techniques for dimensionality reduction is principal
component analysis (PCA). In PCA, the directions with the largest variances are
considered as most “important” (i.e., the most principal) features. Therefore, to find
the most important features, one needs to find the directions of the data that have the
maximum amount of variance. PCA algorithm can be summarized in the following
steps (see Fig. 5.50):
• The first step is to perform standardization (i.e., subtracting the mean and
dividing by the standard deviation).
• The next step is to find the covariance matrix of the features.
# Principal Component
% Variance Explained
Fig. 5.50 An illustration of PCA