368
Shalabh and S. S. Dhar
The second and third columns are providing the lower and upper limits of
prediction interval. For example, lwr = 125.9713 is the lower limit of prediction interval obtained from
p f − t α
2 ,n−k−1
ˆ
σ 2 [1 + x
T
0 (X T X ) −1 x 0 ]
and upr
= 132.9591 indicates the upper limit of prediction interval obtained from
p f + t α
2 ,n−k−1
ˆ
σ 2 [1 + x
T
0 (X T X ) −1 x 0 ]
. This suggests that the predicted value
lies in the interval (125.9713, 132.9591) with 95% chance. Similarly, the second
predicted value for which lwr = 111.0423 and upr = 117.9236 lies in the
interval (111.0423, 117.9236) with 95% chance.
4 Large Number of Explanatory Variables
It is not a debatable issue that large number of factors and variables affect the outcome of any process and suppose that we decide to consider all possible factors and
variables. In such a case, use of a large number of variables triggers its own issues,
e.g., the explanatory power of the variables is distributed among large number of
variables, and so it becomes challenging to identify, which are the crucial variables
in the sense that they contribute more in understanding the model. Sometimes it
is challenging to perform the computations due to large number of variables, e.g.,
inverting the matrix of high order may be difficult, etc. Out of these large numbers of
variables, the question is how to choose those variables, which are more crucial and
contribute more in explaining the model. This objective can be achieved by observing
the values of regression coefficients (β j ’s). If value of any regression coefficient is
minimal, ideally β j = 0, then this means that the rate of change in the average value
of study variable with respect to a unit change in the value of associated jth explanatory variable X j is minimal. This indicates that X j is not contributing significantly in
explaining the behaviour of the model. Hence it is not a relevant variable and can be
dropped from the model.
It is assumed that all the explanatory variables are independent of each other and
correlation between any two variables is ideally zero. This is usually not possible in
practice to achieve and the presence of such correlation increases the variability of
estimates of regression coefficient. Consequently, the model becomes undependable.
This is termed as problem of multicollinearity. One popular approach to estimate the
parameters under the problem of multicollinearity is ridge regression introduced by
Hoerl and Kennard (1970). The idea behind the ridge regression is to impose a penalty
on the regression coefficient and then estimate them. This will help in choosing those
regression coefficients, which are away from zero and thus assisting the modelling
in two ways- obtaining the estimates of regression coefficients and choosing the
irrelevant variables. This concept was extended by Tibshirani (1996) in introducing
the LASSO (least absolute shrinkage and selection operator) regression to choose
those explanatory variables whose corresponding regression coefficients are away
Shalabh and S. S. Dhar
The second and third columns are providing the lower and upper limits of
prediction interval. For example, lwr = 125.9713 is the lower limit of prediction interval obtained from
p f − t α
2 ,n−k−1
ˆ
σ 2 [1 + x
T
0 (X T X ) −1 x 0 ]
and upr
= 132.9591 indicates the upper limit of prediction interval obtained from
p f + t α
2 ,n−k−1
ˆ
σ 2 [1 + x
T
0 (X T X ) −1 x 0 ]
. This suggests that the predicted value
lies in the interval (125.9713, 132.9591) with 95% chance. Similarly, the second
predicted value for which lwr = 111.0423 and upr = 117.9236 lies in the
interval (111.0423, 117.9236) with 95% chance.
4 Large Number of Explanatory Variables
It is not a debatable issue that large number of factors and variables affect the outcome of any process and suppose that we decide to consider all possible factors and
variables. In such a case, use of a large number of variables triggers its own issues,
e.g., the explanatory power of the variables is distributed among large number of
variables, and so it becomes challenging to identify, which are the crucial variables
in the sense that they contribute more in understanding the model. Sometimes it
is challenging to perform the computations due to large number of variables, e.g.,
inverting the matrix of high order may be difficult, etc. Out of these large numbers of
variables, the question is how to choose those variables, which are more crucial and
contribute more in explaining the model. This objective can be achieved by observing
the values of regression coefficients (β j ’s). If value of any regression coefficient is
minimal, ideally β j = 0, then this means that the rate of change in the average value
of study variable with respect to a unit change in the value of associated jth explanatory variable X j is minimal. This indicates that X j is not contributing significantly in
explaining the behaviour of the model. Hence it is not a relevant variable and can be
dropped from the model.
It is assumed that all the explanatory variables are independent of each other and
correlation between any two variables is ideally zero. This is usually not possible in
practice to achieve and the presence of such correlation increases the variability of
estimates of regression coefficient. Consequently, the model becomes undependable.
This is termed as problem of multicollinearity. One popular approach to estimate the
parameters under the problem of multicollinearity is ridge regression introduced by
Hoerl and Kennard (1970). The idea behind the ridge regression is to impose a penalty
on the regression coefficient and then estimate them. This will help in choosing those
regression coefficients, which are away from zero and thus assisting the modelling
in two ways- obtaining the estimates of regression coefficients and choosing the
irrelevant variables. This concept was extended by Tibshirani (1996) in introducing
the LASSO (least absolute shrinkage and selection operator) regression to choose
those explanatory variables whose corresponding regression coefficients are away