Chapter 37
Evaluation of Air Quality Maps Using
Cross-Validation: Metrics, Diagnostics
and Optimization
Richard Ménard and Martin Deshaies-Jacques
Abstract Since 2002, Environment and Climate Change Canada (ECCC) has been
producing hourly surface analyses of pollutants covering North America which have
been used in numerous health impact studies. The analyses are produced using an
optimum interpolation scheme that combines the operational air quality forecast
model GEM-MACH outputs (CHRONOS model outputs were used prior to 2009)
with real-time hourly observations of O 3 , PM 2.5 , PM 10 , NO 2 , and SO 2 . We examine how passive observations, that are not used to create the analysis (i.e. crossvalidation), can be used to evaluate the analysis error and to optimize the input error
statistics.
37.1 Introduction
Maps (or analyses as called in data assimilation) of surface pollutants over Canada
and USA have been produced hourly since 2002 using an optimum interpolation
scheme combining the operational air quality model output of GEM-MACH (or
CHRONOS prior to 2009) with real-time AirNow hourly observations [4, 6, 10].
Maps of surface O 3 and PM 2.5 were thus produced, and PM 10 , NO 2 , NO and SO 2
were added in April 2015 along with a 3-hourly averaged air quality health index
(AQHI) map [11]. Collected over a long period of time, i.e. since 2002, this multiyear
analysis was examined by Robichaud and Ménard [10] and O 3 maps have been used
in several health impact studies [1–3, 12]. What we will discuss here is how to
quantify the error of these analyses.
It is important to remark that to produce these analyses requires the knowledge of
the statistics of the observation and model errors. Furthermore, the observation error
is not simply the instrument error but should also include; errors due to the mismatch
R. Ménard (B) · M. Deshaies-Jacques
Air Quality Research Division, Environment and Climate Change Canada, Dorval H9P 1J3,
Canada
e-mail: Richard.menard@canada.ca
M. Deshaies-Jacques
e-mail: martin.deshaies-jacques@canada.ca
© Crown 2020
C. Mensink et al. (eds.), Air Pollution Modeling and its Application XXVI,
Springer Proceedings in Complexity, https://doi.org/10.1007/978-3-030-22055-6_37
237
Evaluation of Air Quality Maps Using
Cross-Validation: Metrics, Diagnostics
and Optimization
Richard Ménard and Martin Deshaies-Jacques
Abstract Since 2002, Environment and Climate Change Canada (ECCC) has been
producing hourly surface analyses of pollutants covering North America which have
been used in numerous health impact studies. The analyses are produced using an
optimum interpolation scheme that combines the operational air quality forecast
model GEM-MACH outputs (CHRONOS model outputs were used prior to 2009)
with real-time hourly observations of O 3 , PM 2.5 , PM 10 , NO 2 , and SO 2 . We examine how passive observations, that are not used to create the analysis (i.e. crossvalidation), can be used to evaluate the analysis error and to optimize the input error
statistics.
37.1 Introduction
Maps (or analyses as called in data assimilation) of surface pollutants over Canada
and USA have been produced hourly since 2002 using an optimum interpolation
scheme combining the operational air quality model output of GEM-MACH (or
CHRONOS prior to 2009) with real-time AirNow hourly observations [4, 6, 10].
Maps of surface O 3 and PM 2.5 were thus produced, and PM 10 , NO 2 , NO and SO 2
were added in April 2015 along with a 3-hourly averaged air quality health index
(AQHI) map [11]. Collected over a long period of time, i.e. since 2002, this multiyear
analysis was examined by Robichaud and Ménard [10] and O 3 maps have been used
in several health impact studies [1–3, 12]. What we will discuss here is how to
quantify the error of these analyses.
It is important to remark that to produce these analyses requires the knowledge of
the statistics of the observation and model errors. Furthermore, the observation error
is not simply the instrument error but should also include; errors due to the mismatch
R. Ménard (B) · M. Deshaies-Jacques
Air Quality Research Division, Environment and Climate Change Canada, Dorval H9P 1J3,
Canada
e-mail: Richard.menard@canada.ca
M. Deshaies-Jacques
e-mail: martin.deshaies-jacques@canada.ca
© Crown 2020
C. Mensink et al. (eds.), Air Pollution Modeling and its Application XXVI,
Springer Proceedings in Complexity, https://doi.org/10.1007/978-3-030-22055-6_37
237