3 Uncertainty Quantification in Lasso-Type Regularization Problems
95
Fig. 3.6 Coefficient path of the parameters for the Gaia dataset
3.4 Uncertainty Quantification
3.4.1 Refit-LASSO
The “refit”-LASSO is one of the possible ways to quantify system uncertainty of a
LASSO-fitted model. The simple idea is to use the “important” (non-zero) variables
selected by the LASSO procedure in a subsequent OLS fit.
We implement a slight modification of this idea. We carry out the entire crossvalidation procedure multiple times with random partitions, which gives us different
optimized λ for each run, producing an ensemble of possible estimates of β. We
then let the ensemble vote on the inclusion of the variables into the model. We will
consider variables as important, if they have not been shrunk to 0 for a pre-defined
proportion of the runs. Then we apply an OLS fit on the important variables to
get the refit-LASSO estimates. Standard errors of the j ’th parameter estimate, ˆ
β j ,
are then obtained as s
(X T X)
−1
j , where the suffix j indicates the j ’th diagonal
element taken after application of the inverse, and s 2 denotes the unbiased estimator
of σ 2 .