4.3 Discussion of the Validation
83
An important oil drift parameter for seabirds and marine mammals is the exposure
time, i.e. how long harmful oil is present in a grid cell. The oil drift model OSCAR
estimated considerably shorter exposure time in the grid cells than the exposure
time that what was derived from the satellite data (cf. Table 4.6). This difference
is the main explanation for the relatively large difference in estimated seabird and
marine mammal mortality using modelled oil drift data and oil drift data derived
from satellite data in Table 4.7.
The modelled oil drift used as input to the validation study does not include oil
spill response. The effect of the oil spill response on the field-estimated impact for
the two incidents is not known but it is reasonable to assume that the oil spill response
measures implemented during the DHOS reduced the mortality and impacted shoreline area significantly. French-McCay et al. (2018) and Bock et al. (2018) demonstrated that surface oil mass, volume and area were significantly reduced by mechanical recovery, in-situ burning, surface and subsea injection dispersant, and that the
relative risks to shoreline-, surface wildlife- and most aquatic life VECs were reduced
for a hypothetical deep-water oil well blowout in the Gulf of Mexico. In simulations
where shoreline oiling occurred, oil spill response also resulted in less volume ashore
and shorter length of shoreline affected. Including oil spill response in the OSCAR
model, both offshore and in coastal areas, would have reduced ERA Acute calculated
impact on shoreline habitats.
Adequately documenting tests of risk assessment models requires explicit performance criteria against which the model performance is compared (cf. Kirchner et al.
1996; Rykiel 1996). In this study we defined performance criteria based on injury
estimates from incident damage assessments, as well as peer reviewed literature,
from two oil spill cases. This approach was valuable for evaluating the impact estimated by ERA Acute and also for comparing the performance of two alternative
impact functions for the surface compartment (cf. Sect. 3.4.1). However, the performance must be interpreted relative to the width of the boundaries. For instance, the
large uncertainty in injury estimates for seabirds after the DHOS incident increases
the likelihood of obtaining a high-performance score. The estimated average loss of
approximately 150,000 seabirds by ERA Acute was within, but slightly low compared
to the performance boundaries. The studies by Haney et al. (2014a, b) was criticized
by Sackmann et al. (2015) who suggested that an underestimation of carcass transport
probability to shorelines was leading to overestimation of bird deaths by an order
of magnitude; a comment which was refuted in a response letter from Haney et al.
(2015) (see also Beyer et al. 2016). When compared only against the injury estimates
from the DHOS NRDA process, the estimated impact by ERA Acute is somewhat
high (conservative), for impacts estimated using both modeled oil drift data and oil
drift data derived from satellites.
83
An important oil drift parameter for seabirds and marine mammals is the exposure
time, i.e. how long harmful oil is present in a grid cell. The oil drift model OSCAR
estimated considerably shorter exposure time in the grid cells than the exposure
time that what was derived from the satellite data (cf. Table 4.6). This difference
is the main explanation for the relatively large difference in estimated seabird and
marine mammal mortality using modelled oil drift data and oil drift data derived
from satellite data in Table 4.7.
The modelled oil drift used as input to the validation study does not include oil
spill response. The effect of the oil spill response on the field-estimated impact for
the two incidents is not known but it is reasonable to assume that the oil spill response
measures implemented during the DHOS reduced the mortality and impacted shoreline area significantly. French-McCay et al. (2018) and Bock et al. (2018) demonstrated that surface oil mass, volume and area were significantly reduced by mechanical recovery, in-situ burning, surface and subsea injection dispersant, and that the
relative risks to shoreline-, surface wildlife- and most aquatic life VECs were reduced
for a hypothetical deep-water oil well blowout in the Gulf of Mexico. In simulations
where shoreline oiling occurred, oil spill response also resulted in less volume ashore
and shorter length of shoreline affected. Including oil spill response in the OSCAR
model, both offshore and in coastal areas, would have reduced ERA Acute calculated
impact on shoreline habitats.
Adequately documenting tests of risk assessment models requires explicit performance criteria against which the model performance is compared (cf. Kirchner et al.
1996; Rykiel 1996). In this study we defined performance criteria based on injury
estimates from incident damage assessments, as well as peer reviewed literature,
from two oil spill cases. This approach was valuable for evaluating the impact estimated by ERA Acute and also for comparing the performance of two alternative
impact functions for the surface compartment (cf. Sect. 3.4.1). However, the performance must be interpreted relative to the width of the boundaries. For instance, the
large uncertainty in injury estimates for seabirds after the DHOS incident increases
the likelihood of obtaining a high-performance score. The estimated average loss of
approximately 150,000 seabirds by ERA Acute was within, but slightly low compared
to the performance boundaries. The studies by Haney et al. (2014a, b) was criticized
by Sackmann et al. (2015) who suggested that an underestimation of carcass transport
probability to shorelines was leading to overestimation of bird deaths by an order
of magnitude; a comment which was refuted in a response letter from Haney et al.
(2015) (see also Beyer et al. 2016). When compared only against the injury estimates
from the DHOS NRDA process, the estimated impact by ERA Acute is somewhat
high (conservative), for impacts estimated using both modeled oil drift data and oil
drift data derived from satellites.