Statistical Modelling and Variable Selection in Climate Science
353
variables. The solutions of such an approach are algorithm and computational based.
The idea behind the LASSO was motivated by ridge regression in which some penalty
is imposed on the regression coefficients, which can discriminate the regression
coefficients that are close to zero and away from zero.
The statistical software plays a vital role in computing the mathematical functions
and finding out the statistical models. Nowadays, the statistical analysis is performed
with the help of various paid and free software. Among them, the open source R
software (available at www.r-project.org) has gained popularity in the last decade.
An advantage of this software is that it is a free software, and the researchers can
contribute their own created packages, which can be downloaded by other researchers
to use them.
We have addressed and illustrated how to initiate the linear regression modelling
and subset selection using LASSO based on the available set of data. The basic
concepts of linear regression modelling and subset selection through LASSO are
explained and an attempt is made to keep the level of involved mathematics as low
as possible. The objective is to familiarize the users with regression techniques to
enable them to be confident to use it and improve their models. The main steps in
any regression modelling and subset selection using LASSO technique are explained
and illustrated using a small data set using R software. The area of linear regression
analysis is vast and involves many aspects to be considered before arriving at the
final model. Addressing of all such issues is not the aim of this chapter. Many books
are available in literature who will give in-depth knowledge of these topics and the
more interested reader is referred to books by Rao et al. (2008), Montgomery et al.
(2012), Draper and Smith (2014), Heumann et al. (2016) etc.
The plan of this chapter is as follows. The concepts and tools of multiple linear
regression modelling are explained in Sect. 2 and its six subsections. A dataset is
considered and the implementation of the tools developed in Sect. 3 is demonstrated
using the R software. Section 4 discusses the role, issues, and the implications of having a large number of explanatory variables in the model followed by a discussion on
the role of ridge regression in Sect. 5. Section 6 discusses the LASSO regression and
its role in the selection of a subset of “important” explanatory variables followed by
the data-based example using the R software. Last Sect. 7 presents some conclusions
followed by Bibliography.
2 Multiple Linear Regression Modelling
Consider a situation, where the output of a variable, called as dependent variable or
study variable depends upon several input variables, called as covariates, regressors
or explanatory variables and such relationship is linear in nature. Various graphs
between and dependent and independent variables help in confirming the linear
relationship. A model consists of variables and parameters, and finding the model is
equivalent to finding the values of the involved parameters. The technique of linear
353
variables. The solutions of such an approach are algorithm and computational based.
The idea behind the LASSO was motivated by ridge regression in which some penalty
is imposed on the regression coefficients, which can discriminate the regression
coefficients that are close to zero and away from zero.
The statistical software plays a vital role in computing the mathematical functions
and finding out the statistical models. Nowadays, the statistical analysis is performed
with the help of various paid and free software. Among them, the open source R
software (available at www.r-project.org) has gained popularity in the last decade.
An advantage of this software is that it is a free software, and the researchers can
contribute their own created packages, which can be downloaded by other researchers
to use them.
We have addressed and illustrated how to initiate the linear regression modelling
and subset selection using LASSO based on the available set of data. The basic
concepts of linear regression modelling and subset selection through LASSO are
explained and an attempt is made to keep the level of involved mathematics as low
as possible. The objective is to familiarize the users with regression techniques to
enable them to be confident to use it and improve their models. The main steps in
any regression modelling and subset selection using LASSO technique are explained
and illustrated using a small data set using R software. The area of linear regression
analysis is vast and involves many aspects to be considered before arriving at the
final model. Addressing of all such issues is not the aim of this chapter. Many books
are available in literature who will give in-depth knowledge of these topics and the
more interested reader is referred to books by Rao et al. (2008), Montgomery et al.
(2012), Draper and Smith (2014), Heumann et al. (2016) etc.
The plan of this chapter is as follows. The concepts and tools of multiple linear
regression modelling are explained in Sect. 2 and its six subsections. A dataset is
considered and the implementation of the tools developed in Sect. 3 is demonstrated
using the R software. Section 4 discusses the role, issues, and the implications of having a large number of explanatory variables in the model followed by a discussion on
the role of ridge regression in Sect. 5. Section 6 discusses the LASSO regression and
its role in the selection of a subset of “important” explanatory variables followed by
the data-based example using the R software. Last Sect. 7 presents some conclusions
followed by Bibliography.
2 Multiple Linear Regression Modelling
Consider a situation, where the output of a variable, called as dependent variable or
study variable depends upon several input variables, called as covariates, regressors
or explanatory variables and such relationship is linear in nature. Various graphs
between and dependent and independent variables help in confirming the linear
relationship. A model consists of variables and parameters, and finding the model is
equivalent to finding the values of the involved parameters. The technique of linear