74
Y. Ozaki et al.
with light absorption. In some cases, the former brings about more prominent variations to spectra than the latter. The response of the spectral data to the physical
effects is significant baseline variations. On the basis of this study, Geladi et al.
[17] proposed multiplicative scattering collection (MSC) as a preprocessing tool to
correct the light scattering problems in the NIR spectra.
This section explains four kinds of data pretreatment methods, noise reduction methods, baseline correction methods, resolution enhancement methods, and
centering and normalization methods [1–3].
4.3.1 Noise Reduction Methods
In NIR spectroscopy, several kinds of noise are caused by a variety of interfering
physical and/or chemical process [1–3]. The most general noise is high-frequency
noise associated with the instrument’s detector and electronic circuits. There are
other forms of noise as well; for example, low-frequency noise and localized noise.
Low-frequency noise is induced, for instance, by instrument drift during the scanning measurements. The reduction of the low-frequency noise may be more difficult
because it often resembles the real information in the data.
Most standard method to improve SN ratio in spectra is accumulation-average
processing that requires to increase the accumulation number and calculate an
average. This reduces the effects of high-frequency noise significantly, but technically it is not a “pretreatment” but a normal, integrated part of collecting spectra. If
the noise reduction by the accumulation average is still insufficient, one can employ
smoothing to remove high-frequency noise. The most commonly used smoothing
methods are moving-average method and Savitzky–Golay method [1, 2].
The moving-average method is the simplest type of smoothing [1–3]. In this
method, the reading A i ’ (A is, for instance, absorbance) at each variable i = 1, 2, ---k
is replaced by a weighted average of itself and its nearest neighbors. From i−n to
i+n:
A i =
n
k=−n
w k A i+k
(4.3)
w k , defining the smoothing, is called the convolution weights.
The Savitzky–Golay method originated from the idea that in the vicinity of a
measurement point a spectrum can be fitted by low-degree polynomials [18]. Practically, w k is determined by fitting the spectrum with low-degree polynomials using
least squares regression. Savitzky and Golay calculated w k for the different orders
of polynomials and N (N = 2n+1) [18]. One can find these calculated convolution
weights in a numeral table. For example, when N is equal to 5, smoothed values can
Y. Ozaki et al.
with light absorption. In some cases, the former brings about more prominent variations to spectra than the latter. The response of the spectral data to the physical
effects is significant baseline variations. On the basis of this study, Geladi et al.
[17] proposed multiplicative scattering collection (MSC) as a preprocessing tool to
correct the light scattering problems in the NIR spectra.
This section explains four kinds of data pretreatment methods, noise reduction methods, baseline correction methods, resolution enhancement methods, and
centering and normalization methods [1–3].
4.3.1 Noise Reduction Methods
In NIR spectroscopy, several kinds of noise are caused by a variety of interfering
physical and/or chemical process [1–3]. The most general noise is high-frequency
noise associated with the instrument’s detector and electronic circuits. There are
other forms of noise as well; for example, low-frequency noise and localized noise.
Low-frequency noise is induced, for instance, by instrument drift during the scanning measurements. The reduction of the low-frequency noise may be more difficult
because it often resembles the real information in the data.
Most standard method to improve SN ratio in spectra is accumulation-average
processing that requires to increase the accumulation number and calculate an
average. This reduces the effects of high-frequency noise significantly, but technically it is not a “pretreatment” but a normal, integrated part of collecting spectra. If
the noise reduction by the accumulation average is still insufficient, one can employ
smoothing to remove high-frequency noise. The most commonly used smoothing
methods are moving-average method and Savitzky–Golay method [1, 2].
The moving-average method is the simplest type of smoothing [1–3]. In this
method, the reading A i ’ (A is, for instance, absorbance) at each variable i = 1, 2, ---k
is replaced by a weighted average of itself and its nearest neighbors. From i−n to
i+n:
A i =
n
k=−n
w k A i+k
(4.3)
w k , defining the smoothing, is called the convolution weights.
The Savitzky–Golay method originated from the idea that in the vicinity of a
measurement point a spectrum can be fitted by low-degree polynomials [18]. Practically, w k is determined by fitting the spectrum with low-degree polynomials using
least squares regression. Savitzky and Golay calculated w k for the different orders
of polynomials and N (N = 2n+1) [18]. One can find these calculated convolution
weights in a numeral table. For example, when N is equal to 5, smoothed values can