358
Shalabh and S. S. Dhar
The estimators ˆ
β in (7) and ˆ
σ
2 in (10) are the point estimators of β and σ
2
respectively based on ordinary least squares estimation. They are called as ordinary
least squares estimators of β and σ
2 .
Just like the ordinary least squares estimation, there is another method to estimate
the unknown parameters, called as maximum likelihood estimation. If the maximum
likelihood estimation method is employed to estimate β and σ
2 based on given
observations on y and X, then the maximum likelihood estimates of β and σ
2 are
obtained as
˜
β = (X
T X )
−1 X
T Y
(13)
and
˜
σ
2
=
(y − X ˆ
β)
T
(y − X ˆ
β)
n
=
y
T
[I − X (X
T X )
−1 X
T
]y
n
,
(14)
respectively. Note that the ordinary least squares and maximum likelihood estimates
of β are the same whereas the ordinary least squares and maximum likelihood estimates of σ
2 are different. Moreover, the maximum likelihood estimates of β and
σ
2 in (13) and (14), respectively remain valid as long as the probability distribution
of random error component in (3) remains multivariate normal. If this distribution
changes, the maximum likelihood estimates will change.
After obtaining the values of the regression coefficients based on given data,
the fitted model is obtained as y = X ˆ
β. Next, by substituting the values of given
explanatory variables in the model y = X ˆ
β, the values of study variable are obtained
as ˆ
y = X ˆ
β, which are called as fitted values. The fitted values indicate that these values would have been the values of study variable if the values of estimated parameters
had been the actual values of the parameters. The difference between the observed
values y and fitted values ˆ
y is called as residual given by e = y ∼ ˆ
y (symbol ~
denotes the difference). Usually, we consider e = y − ˆ
y = y − X ˆ
β. In an ideal
model, one would expect the residuals to be zero. Residuals help in checking various model assumptions about ε based on a given sample of data. Here ˆ
β and ˆ
σ
2
(or equivalently ˜
β and ˜
σ
2 ) are the point estimates of β and σ
2 , respectively as they
provide the values at points, i.e., the single values.
2.3 Confidence Interval Estimation
The interval estimation provides the value of parameters in an interval, in terms of
confidence intervals. The confidence intervals in multiple regression model can be
constructed for the individual as well as joint regression coefficients. We consider
both of them as follows:
The 100(1 − α)% confidence interval or equivalently the interval estimate of
individual regression coefficients β j ( j = 0, 1, 2, . . . , k) is given by
Shalabh and S. S. Dhar
The estimators ˆ
β in (7) and ˆ
σ
2 in (10) are the point estimators of β and σ
2
respectively based on ordinary least squares estimation. They are called as ordinary
least squares estimators of β and σ
2 .
Just like the ordinary least squares estimation, there is another method to estimate
the unknown parameters, called as maximum likelihood estimation. If the maximum
likelihood estimation method is employed to estimate β and σ
2 based on given
observations on y and X, then the maximum likelihood estimates of β and σ
2 are
obtained as
˜
β = (X
T X )
−1 X
T Y
(13)
and
˜
σ
2
=
(y − X ˆ
β)
T
(y − X ˆ
β)
n
=
y
T
[I − X (X
T X )
−1 X
T
]y
n
,
(14)
respectively. Note that the ordinary least squares and maximum likelihood estimates
of β are the same whereas the ordinary least squares and maximum likelihood estimates of σ
2 are different. Moreover, the maximum likelihood estimates of β and
σ
2 in (13) and (14), respectively remain valid as long as the probability distribution
of random error component in (3) remains multivariate normal. If this distribution
changes, the maximum likelihood estimates will change.
After obtaining the values of the regression coefficients based on given data,
the fitted model is obtained as y = X ˆ
β. Next, by substituting the values of given
explanatory variables in the model y = X ˆ
β, the values of study variable are obtained
as ˆ
y = X ˆ
β, which are called as fitted values. The fitted values indicate that these values would have been the values of study variable if the values of estimated parameters
had been the actual values of the parameters. The difference between the observed
values y and fitted values ˆ
y is called as residual given by e = y ∼ ˆ
y (symbol ~
denotes the difference). Usually, we consider e = y − ˆ
y = y − X ˆ
β. In an ideal
model, one would expect the residuals to be zero. Residuals help in checking various model assumptions about ε based on a given sample of data. Here ˆ
β and ˆ
σ
2
(or equivalently ˜
β and ˜
σ
2 ) are the point estimates of β and σ
2 , respectively as they
provide the values at points, i.e., the single values.
2.3 Confidence Interval Estimation
The interval estimation provides the value of parameters in an interval, in terms of
confidence intervals. The confidence intervals in multiple regression model can be
constructed for the individual as well as joint regression coefficients. We consider
both of them as follows:
The 100(1 − α)% confidence interval or equivalently the interval estimate of
individual regression coefficients β j ( j = 0, 1, 2, . . . , k) is given by