188
Chapter 10: The Evaluation of Forecasts
Unfortunately, this new score is inequitable, because it is higher for a11
forecasts of near normal (n2j = ~, n1j = n3j = 0, j = 1,2,3) than for a11 of
either of the other two categories. To fix this the second row in Table 10.2b
is adjusted so that the row elements sum to zero (as they already do for the
first and third rows) while preserving the differences among them, i.e. by
subtracting
1 3
3" L 5 2j = 0.33
j=1
from each 52j. This ensures that H in (10.2) will sum to zero when every
forecast is for near normal, for below or above. The result of this operation is
shown in Table 10.2c. Note, however, that now for perfect forecasts (nij = ~
for i = j and = 0 for i - : : P j)
T 3
H=- LSii-::PT
3 i=1
and therefore, S < 1. This is caused by the reduction of the near normal hit
reward.
Thus, in a final transformation (Table 10.2d) diagonal elements in the
weighting table are proportionately adjusted to sum to 3 so that H = T for
perfect forecasts, and then rows are readjusted so
3
I > i j = 0
j=1
for a11 i to ensure H = 0 for single category forecast sets. This weighting
scheme preserves Heidke's equitability and scaling as we11 as the linearly
weighted scheme's sma11er difference from the correlation score and relative
insensitivity to the number of categories.
c) Linear Error in Probability Space (LEPS)
While the refinements of the Heidke score described above were weH motivated, they were nevertheless implemented arbitrarily. In contrast, the family
of LEPS scoring mIes (Ward and Folland, 1991) a11 have a rational basis and
fo11ow from a score
S='1-IFp -Fol
where IF p - Fol denotes the difference in cumulative probabilities of a forecast and verifying observation. This leads eventually to scoring schemes for
continuous forecasts (LEPSCONT), prob ability forecasts (LEPSPROB), and
categorical forecasts (LEPSCAT). Only the latter will be presented here.
The scoring mIes for LEPSCAT with three equi-probable classes are displayed in Table 10.3 (again from Barnston, 1992, who also has worked out
Chapter 10: The Evaluation of Forecasts
Unfortunately, this new score is inequitable, because it is higher for a11
forecasts of near normal (n2j = ~, n1j = n3j = 0, j = 1,2,3) than for a11 of
either of the other two categories. To fix this the second row in Table 10.2b
is adjusted so that the row elements sum to zero (as they already do for the
first and third rows) while preserving the differences among them, i.e. by
subtracting
1 3
3" L 5 2j = 0.33
j=1
from each 52j. This ensures that H in (10.2) will sum to zero when every
forecast is for near normal, for below or above. The result of this operation is
shown in Table 10.2c. Note, however, that now for perfect forecasts (nij = ~
for i = j and = 0 for i - : : P j)
T 3
H=- LSii-::PT
3 i=1
and therefore, S < 1. This is caused by the reduction of the near normal hit
reward.
Thus, in a final transformation (Table 10.2d) diagonal elements in the
weighting table are proportionately adjusted to sum to 3 so that H = T for
perfect forecasts, and then rows are readjusted so
3
I > i j = 0
j=1
for a11 i to ensure H = 0 for single category forecast sets. This weighting
scheme preserves Heidke's equitability and scaling as we11 as the linearly
weighted scheme's sma11er difference from the correlation score and relative
insensitivity to the number of categories.
c) Linear Error in Probability Space (LEPS)
While the refinements of the Heidke score described above were weH motivated, they were nevertheless implemented arbitrarily. In contrast, the family
of LEPS scoring mIes (Ward and Folland, 1991) a11 have a rational basis and
fo11ow from a score
S='1-IFp -Fol
where IF p - Fol denotes the difference in cumulative probabilities of a forecast and verifying observation. This leads eventually to scoring schemes for
continuous forecasts (LEPSCONT), prob ability forecasts (LEPSPROB), and
categorical forecasts (LEPSCAT). Only the latter will be presented here.
The scoring mIes for LEPSCAT with three equi-probable classes are displayed in Table 10.3 (again from Barnston, 1992, who also has worked out