Chapter 3
Uncertainty Quantification in Lasso-Type
Regularization Problems
Tathagata Basu, Jochen Einbeck, and Matthias C. M. Troffaes
Abstract Regularization techniques, which sit at the interface of statistical modeling and machine learning, are often used in the engineering or other applied
sciences to tackle high dimensional regression (type) problems. While a number
of regularization methods are commonly used, the ‘Least Absolute Shrinkage and
Selection Operator’ or simply LASSO is popular because of its efficient variable
selection property. This property of the LASSO helps to deal with problems where
the number of predictors is larger than the total number of observations, as it
shrinks the coefficients of non-important parameters to zero. In this chapter, both
frequentist and Bayesian approaches for the LASSO are discussed, with particular
attention to the problem of uncertainty quantification of regression parameters.
For the frequentist approach, we discuss a refit technique as well as the classical
bootstrap method, and for the Bayesian method, we make use of the equivalent
LASSO formulation using a Laplace prior on the model parameters.
Keywords Statistical modeling · LASSO · Bayesian statistics · Uncertainty
quantification
3.1 Introduction
Statistics is a collection of mathematical concepts to analyze and find the structure
in data. Data can be either numeric- or character-valued (representing a class)
depending on the problem. There are several purposes of statistics; however one
of the main purposes is description of the data and prediction of system behavior
from the observed data. Elements of statistical reasoning have been traced back as
early as 400 AD [14, p. 7] in India. However, the modern-day approach only started
T. Basu () · J. Einbeck · M. C. M. Troffaes
Durham University, Durham, UK
e-mail: tathagata.basu@durham.ac.uk; jochen.einbeck@durham.ac.uk;
matthias.troffaes@durham.ac.uk
© Springer Nature Switzerland AG 2021
M. Vasile (ed.), Optimization Under Uncertainty with Applications to Aerospace
Engineering, https://doi.org/10.1007/978-3-030-60166-9_3
81
Uncertainty Quantification in Lasso-Type
Regularization Problems
Tathagata Basu, Jochen Einbeck, and Matthias C. M. Troffaes
Abstract Regularization techniques, which sit at the interface of statistical modeling and machine learning, are often used in the engineering or other applied
sciences to tackle high dimensional regression (type) problems. While a number
of regularization methods are commonly used, the ‘Least Absolute Shrinkage and
Selection Operator’ or simply LASSO is popular because of its efficient variable
selection property. This property of the LASSO helps to deal with problems where
the number of predictors is larger than the total number of observations, as it
shrinks the coefficients of non-important parameters to zero. In this chapter, both
frequentist and Bayesian approaches for the LASSO are discussed, with particular
attention to the problem of uncertainty quantification of regression parameters.
For the frequentist approach, we discuss a refit technique as well as the classical
bootstrap method, and for the Bayesian method, we make use of the equivalent
LASSO formulation using a Laplace prior on the model parameters.
Keywords Statistical modeling · LASSO · Bayesian statistics · Uncertainty
quantification
3.1 Introduction
Statistics is a collection of mathematical concepts to analyze and find the structure
in data. Data can be either numeric- or character-valued (representing a class)
depending on the problem. There are several purposes of statistics; however one
of the main purposes is description of the data and prediction of system behavior
from the observed data. Elements of statistical reasoning have been traced back as
early as 400 AD [14, p. 7] in India. However, the modern-day approach only started
T. Basu () · J. Einbeck · M. C. M. Troffaes
Durham University, Durham, UK
e-mail: tathagata.basu@durham.ac.uk; jochen.einbeck@durham.ac.uk;
matthias.troffaes@durham.ac.uk
© Springer Nature Switzerland AG 2021
M. Vasile (ed.), Optimization Under Uncertainty with Applications to Aerospace
Engineering, https://doi.org/10.1007/978-3-030-60166-9_3
81