7 NIR Data Exploration and Regression by Chemometrics—A Primer
137
Fig. 7.8 Removal of bad
(noisy) region. Including bad
variables in any subsequent
analysis will not serve any
purpose and in worst cases
lead to deterioration of the
multivariate models.
(© Newlin & Engelsen)
principles (e.g., InGaAs). The whole NIR region is situated in a sweet spot in the electromagnetic spectrum, where both the molecular absorption and the scatter intensity
are relatively low ([1] Fig. 7.7). However, different applications may require information from different subregions in the NIR spectral region and if for example the
long-wavelength region is too strongly absorbing (noisy with no or too few photons
reaching the detector) it should be removed prior to further analysis (Fig. 7.8).
Depending on the distribution of particle and microstructure sizes and density,
scatter is perhaps the strongest effect that needs to be removed from the NIR signal.
The scattering effect can be both frequency-dependent (proportional to λ
−4 for
Rayleigh scattering) and dependent on particle size and/or shape (Mie–Lorentz scattering). An example of scattering of data recorded on crystalline sugar powders is
shown in Fig. 7.9. As the granular size changes, the spectra show offsets for the
longer wavelengths, resulting in large deviations in the apparent absorption. Note
that the samples have exactly the same chemistry—only the crystal size is different.
If extreme variations, like the ones observed in Fig. 7.9, are observed in an NIR
spectral ensemble, the pre-processing is likely to fail. But if the scatter variations are
of a lesser magnitude, the pre-processing will often be able to eliminate the apparent
differences in absorption.
7.2.1 Multiplicative Scatter Correction (MSC)
The multiplicative scatter correction (MSC) method was introduced by Martens and
coworkers [12, 13], and together with the standard normal variate (SNV) method [14]
it is the most widely applied NIR pre-processing technique. MSC removes frequencylinear imperfections, both additive and multiplicative, by fitting a first-order function
between the recorded spectra x org and a reference spectrum x ref of the form:
x org = b 0 + b 1 · x ref + e
(7.8)
137
Fig. 7.8 Removal of bad
(noisy) region. Including bad
variables in any subsequent
analysis will not serve any
purpose and in worst cases
lead to deterioration of the
multivariate models.
(© Newlin & Engelsen)
principles (e.g., InGaAs). The whole NIR region is situated in a sweet spot in the electromagnetic spectrum, where both the molecular absorption and the scatter intensity
are relatively low ([1] Fig. 7.7). However, different applications may require information from different subregions in the NIR spectral region and if for example the
long-wavelength region is too strongly absorbing (noisy with no or too few photons
reaching the detector) it should be removed prior to further analysis (Fig. 7.8).
Depending on the distribution of particle and microstructure sizes and density,
scatter is perhaps the strongest effect that needs to be removed from the NIR signal.
The scattering effect can be both frequency-dependent (proportional to λ
−4 for
Rayleigh scattering) and dependent on particle size and/or shape (Mie–Lorentz scattering). An example of scattering of data recorded on crystalline sugar powders is
shown in Fig. 7.9. As the granular size changes, the spectra show offsets for the
longer wavelengths, resulting in large deviations in the apparent absorption. Note
that the samples have exactly the same chemistry—only the crystal size is different.
If extreme variations, like the ones observed in Fig. 7.9, are observed in an NIR
spectral ensemble, the pre-processing is likely to fail. But if the scatter variations are
of a lesser magnitude, the pre-processing will often be able to eliminate the apparent
differences in absorption.
7.2.1 Multiplicative Scatter Correction (MSC)
The multiplicative scatter correction (MSC) method was introduced by Martens and
coworkers [12, 13], and together with the standard normal variate (SNV) method [14]
it is the most widely applied NIR pre-processing technique. MSC removes frequencylinear imperfections, both additive and multiplicative, by fitting a first-order function
between the recorded spectra x org and a reference spectrum x ref of the form:
x org = b 0 + b 1 · x ref + e
(7.8)