372
DAVID ANDERSON ET AL
systems and the coupled forecast model than that used by Vialard et al, the
spread is considerably smaller than the rms error, indicating a problem in the
growth of error versus the growth of spread in the ensemble is that the
forecasts are too confident, indicating that all the uncertainty in the forecasts
is not being accounted for. There is another more optimistic interpretation:
that the spread is the theoretical measure of predictability. If the model error
were small and the initial conditions were correct, then this is how the
forecast error should grow. This estimate can be model dependent so it is
not a hard argument. In practice, it is likely that by improving the models
and the initial conditions, the error growth can be reduced and by improving
the ensemble generation the spread can be increased. At ECMWF we are
developing a multi-model forecasting facility in collaboration with the UK
and French Met Offices and in this multi-model system the separation
between error growth and spread is reduced. See also Palmer et al 2004.
3.5 Problems with the winds
In section 1, we considered the importance of atmospheric reanalyses.
There we showed that the medium-range forecasts from say 20 years ago are
now much better than they were then, partly because of improvements in the
current analysis system compared with what was done then. That is
encouraging but is not particularly useful in its own right, unless one wants
to use these past forecasts for calibration. Calibration on past events has not
really taken hold in the medium range community, though some moves in
this direction are afoot (Lalaurette 2003). But calibration on past events is a
major feature of seasonal and monthly forecast systems. This is described in
Stockdale et al 1998. When making forecasts to a few days ahead, there are
plenty of cases on which to test the model forecasts. When making forecasts
to 6 months ahead, there are very few cases of events such as El Nino that
can be tested in real-time. To evaluate such a system, one has to go back in
time and to make hindcasts from as far back as one can reasonably go. As
mentioned earlier, there are insufficient ocean observations to make ocean
analyses directly. But by using the forcing fields from the atmospheric
reanalyses one can produce ocean initial conditions back say 15 or 20 years.
There is a further reason for using these past hindcasts. All models have
errors. For forecasts out to a few days, these have largely been ignored
though realization that the model would benefit from calibration on past
events is growing. For monthly and seasonal forecasts, the error is
sufficiently large that it can not be ignored. To first order it is estimated
from the past hindcasts and this information is used to calibrate the real-time
forecasts. The atmospheric reanalyses are therefore very important in
enabling ocean reanalyses to be performed. It seems that the ERA40
reanalyses are considerably better than the earlier ERA-15 reanalyses in that
analysis/forecast system. At this stage we do not know if this is due to the analysis or the forecast model. One way of interpreting the difference in the
DAVID ANDERSON ET AL
systems and the coupled forecast model than that used by Vialard et al, the
spread is considerably smaller than the rms error, indicating a problem in the
growth of error versus the growth of spread in the ensemble is that the
forecasts are too confident, indicating that all the uncertainty in the forecasts
is not being accounted for. There is another more optimistic interpretation:
that the spread is the theoretical measure of predictability. If the model error
were small and the initial conditions were correct, then this is how the
forecast error should grow. This estimate can be model dependent so it is
not a hard argument. In practice, it is likely that by improving the models
and the initial conditions, the error growth can be reduced and by improving
the ensemble generation the spread can be increased. At ECMWF we are
developing a multi-model forecasting facility in collaboration with the UK
and French Met Offices and in this multi-model system the separation
between error growth and spread is reduced. See also Palmer et al 2004.
3.5 Problems with the winds
In section 1, we considered the importance of atmospheric reanalyses.
There we showed that the medium-range forecasts from say 20 years ago are
now much better than they were then, partly because of improvements in the
current analysis system compared with what was done then. That is
encouraging but is not particularly useful in its own right, unless one wants
to use these past forecasts for calibration. Calibration on past events has not
really taken hold in the medium range community, though some moves in
this direction are afoot (Lalaurette 2003). But calibration on past events is a
major feature of seasonal and monthly forecast systems. This is described in
Stockdale et al 1998. When making forecasts to a few days ahead, there are
plenty of cases on which to test the model forecasts. When making forecasts
to 6 months ahead, there are very few cases of events such as El Nino that
can be tested in real-time. To evaluate such a system, one has to go back in
time and to make hindcasts from as far back as one can reasonably go. As
mentioned earlier, there are insufficient ocean observations to make ocean
analyses directly. But by using the forcing fields from the atmospheric
reanalyses one can produce ocean initial conditions back say 15 or 20 years.
There is a further reason for using these past hindcasts. All models have
errors. For forecasts out to a few days, these have largely been ignored
though realization that the model would benefit from calibration on past
events is growing. For monthly and seasonal forecasts, the error is
sufficiently large that it can not be ignored. To first order it is estimated
from the past hindcasts and this information is used to calibrate the real-time
forecasts. The atmospheric reanalyses are therefore very important in
enabling ocean reanalyses to be performed. It seems that the ERA40
reanalyses are considerably better than the earlier ERA-15 reanalyses in that
analysis/forecast system. At this stage we do not know if this is due to the analysis or the forecast model. One way of interpreting the difference in the