Statistical Modelling and Variable Selection in Climate Science
357
for the given observations on y and X, and this provides a point estimator of β,
denoted by ˆ
β as
ˆ
β = (X
T X )
−1 X
T y,
(7)
where ˆ
β = ( ˆ
β 0 , ˆ
β 1 , ˆ
β 2 , . . . , ˆ
β k )
T is a vector that provides numerical values to
unknown β = (β 0 , β 1 , β 2 , . . . , β k )
T using the observations y = (y 1 , y 2 , . . . , y n )
T
and X =
⎛
⎜
⎜
⎜
⎝
1 x 11 x 12 · · · x 1k
1 x 21 x 22 · · · x 2k
. . .
. . .
. . .
. . .
. . .
1 x n1 x n2 · · · x nk
⎞
⎟
⎟
⎟
⎠
. This is called as ordinary least squares estimator
(OLSE) of β.
The estimator ˆ
β is an unbiased estimator of β in the sense that
E( ˆ
β) = β,
(8)
and its covariance matrix is given by
V ( ˆ
β) = E( ˆ
β − β)( ˆ
β − β)
T
= σ
2
(X
T X )
−1
.
(9)
Note that the covariance matrix (9) depends upon σ
2 , which is unknown. So it is
estimated as
ˆ
σ
2
=
(y − X ˆ
β)
T
(y − X ˆ
β)
n − (k + 1)
=
y
T
[I − X (X
T X )
−1 X
T
]y
n − (k + 1)
(10)
Note that the meaning of the “estimated” is that the value of the concerned quantity
can be found using the available observations. Then the covariance matrix (9) is
estimated as
V ( ˆ
β)
= ˆ
σ
2
(X
T X )
−1
.
(11)
Let C j j and C j j denote the diagonal and off-diagonal elements of (X
T X )
−1 ,
respectively. The diagonal elements of matrix in (11) provides the values of estimated
variances of ˆ
β j ’s, j = 0, 1, 2, …, k. The off-diagonal elements in matrix in (11) provide
the values of estimated covariance cov( ˆ
β j , ˆ
β j ), j = j
= 1, 2, . . . , k. The positive
square root of values of estimated variances of ˆ
β j ’s, j = 0, 1, 2, …, k is called as
their standard errors and is denoted by se( ˆ
β j ) =
ˆ
σ 2 C j j , j = 0, 1, 2, . . . , k. The
standard error gives an idea about the variation of estimates of ˆ
β j ’s. Smaller the value
of standard errors, better is the estimator.
The estimator ˆ
σ
2 is an unbiased estimator of σ
2 in the sense that
E( ˆ
σ
2
) = σ
2
.
(12)