4.3.3 Accuracy Assessment
Accuracy assessment requires unbiased design, strict sampling procedures, and
rigorous analysis of the classification results to make the accuracy itself reliable
(Congalton and Green 1999). Some factors and issues need to be considered when
performing accuracy assessment, which are reference data selection, sample size,
sampling schemes, assessment techniques (confusion error matrix), etc. (Congalton
1991). Reference data were selected on Landsat images based on visual interpretation of the high resolution orthoimages to identify the pixel type. As for the
sample size, it has been minimized to reduce the cost and time, and also has a large
enough to generate an appropriate error matrix (Congalton and Green 1999). A
general guideline is to collect a minimum of 50 samples for each land cover type
(Congalton and Green 1999). In this study, a more reliable sample size determination method Thompson (1992) was used. When investigating the accuracy of multiclass classification map, sample size can be calculated by:
N ¼
BΠ i 1 À Π i
ð
Þ
b i
2
ð4:1Þ
where N is the sample size; Π i is the proportion of the ith class out of all classes that
has the proportion closest to 50 %; b i is the desired precision; B is determined from
the chi-squared (χ
2 ) table that B is the upper (α/k) Â 100th percentile of the χ
2
distribution with one degree of freedom; α is the allowable probability of error; and
k is the number of classes. When determine the sample size before performing
accuracy assessment, the allowable probability of error α should be determined first
(Thompson 1992). 100 %(1-α) is called the confidence interval. Confidence interval
is a very important parameter of estimating the sample size, because generally it
shows the reliability of an estimation (Thompson 1992). In this study, α and b i was
set to be 0.05 and 0.05 respectively. χ
2 value that used to determine B was
1À ¼ 0:99375. Then B was obtained as 7.568 from the chi-squared table. Since
the values of Π i varied from image to image of different years, 2006 classification
map was taken as an example here. The value of Π i was 39 %. Then sample size can
be determined as 720.
Sampling scheme is another important factor that needs to be considered before
accuracy assessment. Stratified random sampling scheme was employed in this
study to ensure that sufficient samples can be selected for each class. This method
considers classes as strata; then certain number of sample points can be selected
randomly without bias within each stratum (Thompson 1992).
Table 4.4 Accuracy assessment of multi-temporal classification maps
1984
1990
1996
2002
2008
2013
Overall accuracy (%)
90.37
92.90
88.67
88.55
92.08
92.84
76
A. Fu et al.
Accuracy assessment requires unbiased design, strict sampling procedures, and
rigorous analysis of the classification results to make the accuracy itself reliable
(Congalton and Green 1999). Some factors and issues need to be considered when
performing accuracy assessment, which are reference data selection, sample size,
sampling schemes, assessment techniques (confusion error matrix), etc. (Congalton
1991). Reference data were selected on Landsat images based on visual interpretation of the high resolution orthoimages to identify the pixel type. As for the
sample size, it has been minimized to reduce the cost and time, and also has a large
enough to generate an appropriate error matrix (Congalton and Green 1999). A
general guideline is to collect a minimum of 50 samples for each land cover type
(Congalton and Green 1999). In this study, a more reliable sample size determination method Thompson (1992) was used. When investigating the accuracy of multiclass classification map, sample size can be calculated by:
N ¼
BΠ i 1 À Π i
ð
Þ
b i
2
ð4:1Þ
where N is the sample size; Π i is the proportion of the ith class out of all classes that
has the proportion closest to 50 %; b i is the desired precision; B is determined from
the chi-squared (χ
2 ) table that B is the upper (α/k) Â 100th percentile of the χ
2
distribution with one degree of freedom; α is the allowable probability of error; and
k is the number of classes. When determine the sample size before performing
accuracy assessment, the allowable probability of error α should be determined first
(Thompson 1992). 100 %(1-α) is called the confidence interval. Confidence interval
is a very important parameter of estimating the sample size, because generally it
shows the reliability of an estimation (Thompson 1992). In this study, α and b i was
set to be 0.05 and 0.05 respectively. χ
2 value that used to determine B was
1À ¼ 0:99375. Then B was obtained as 7.568 from the chi-squared table. Since
the values of Π i varied from image to image of different years, 2006 classification
map was taken as an example here. The value of Π i was 39 %. Then sample size can
be determined as 720.
Sampling scheme is another important factor that needs to be considered before
accuracy assessment. Stratified random sampling scheme was employed in this
study to ensure that sufficient samples can be selected for each class. This method
considers classes as strata; then certain number of sample points can be selected
randomly without bias within each stratum (Thompson 1992).
Table 4.4 Accuracy assessment of multi-temporal classification maps
1984
1990
1996
2002
2008
2013
Overall accuracy (%)
90.37
92.90
88.67
88.55
92.08
92.84
76
A. Fu et al.