250
F. Firouzi et al.
are covariance. Note that a covariance matrix is a symmetric matrix. The following
matrix demonstrates the covariance matrix of three variables:
⎡
⎣
var 1 var 1,2 var 1,3
var 1,2 var 2 var 2,3
var 1,3 var 2,3 var 3
⎤
⎦
5.1.3 Review of Linear Algebra
Linear algebra is a key foundation to the field of machine learning, and it mostly
discusses vectors, matrices, and linear transformations. In this section, we briefly
overview the fundamentals of linear algebra.
A matrix is a rectangular array of numbers organized in columns and rows.
Numbers appear in a matrix are called entries or elements which can be addressed
by their corresponding row number and column number. The number of rows and
columns is called dimension or order of the matrix. For example, the order of the
following matrix is 2 ∗ 3:
A =
1 3 3
2 5 9
The transpose of a matrix is a new matrix whose row and column indices are
switched/flipped as shown by the following example. Note that the transpose of
a matrix is usually represented by A T or A
:
A
T
=
⎡
⎣
1 2
3 5
2 9
⎤
⎦
A vector is a matrix that has only one column as illustrated below:
v = [2 0 6]
A square matrix is a matrix where the number of its columns is equal to the number
of its rows. A symmetric matrix is a matrix whose transpose is equal to itself:
A = A
T
=
1 2
2 3
A diagonal matrix is a square matrix whose off-diagonal elements are equal to zero:
A = A
T
=
1 0
0 3