252
F. Firouzi et al.
Scaling Matrix
Original Data Points
(Circle)
Rotation Matrix
Shear Matrix
Symmetric Matrix
Fig. 5.3 A visual illustration of matrix transformation
Eigenvector
Normal vector
Fig. 5.4 An illustration of eigenvectors. In contrast to normal vectors, the directions of eigenvector
do not change when a linear transformation is applied
All symmetric matrices (such as covariance matrix) can be decomposed into
three matrices (i.e., rotate, scale, and rotate) as illustrated below:
cov = V DV
T
where V is an orthogonal matrix whose columns are the eigenvectors of the
covariance matrix, and matrix D is a diagonal matrix whose elements are the
corresponding eigenvalues. An eigenvector is a vector that changes by only a scalar
factor and whose directions do not change when a linear transformation is applied
(see Fig. 5.4). Eigenvector and eigenvalue can be defined formally by the following
equation:
Av = λv
In the above equation, A is a transformation matrix, v is a column vector that
represents the eigenvectors of the matrix A, and finally λ is a scalar known as the
eigenvalue.