Overview of Image Processing
77
OA, PA and UA do not take into account the agreement between the data sets
(i. e. classified image and reference data) that arises due to chance alone. Thus,
these measures tend to overestimate the classification accuracy (Ma and Redmond 1995). The kappa coefficient of agreement (K) has the ability to account
for chance agreement (Rosenfeld and Fitzpatrick-Lins 1986). The proportion
of agreement by chance is the result of the misclassifications represented by the
off-diagonal elements of the error matrix. Therefore, K uses all the elements
of the error matrix, and not just the diagonal elements (as is the case with
OA). When some classes have more confusion than others, weighted kappa
may be implemented since it does not treat all the misclassifications (disagreements) equally and tends to give more weight to the confusions that are more
serious than others (Cohen 1968; Naesset 1996). To determine the accuracy
of individual classes, a conditional kappa may be computed (Rosenfeld and
Fitzpatrick-Lins 1986).
It may thus be seen that there are a number of measures that may be computed from an error matrix. Each measure may however be based on different
assumptions about the data and thus may evaluate different components of
accuracy (Lark 1995; Stehman 1999). Therefore, in general, it may be expedient to provide an error matrix with the classified image and report more
than one measure of accuracy to fully describe the quality of that classification
(Stehman 1997).
The error matrix based measures inherently assume that each pixel is associated with only one class in the classified image and only one class in the reference data. Use of these measures to assess the accuracy of fuzzy classification
may therefore under or over estimate the accuracy since a fuzzy classification
has to be degraded to adhere to this assumption. Moreover, in addition to the
classification output, often ambiguities exist in the reference data, which may
therefore be treated as fuzzy and thus other alternative measures need to be
adopted (Foody 1995a; Foody 1995b).
One of the simplest measures to assess the accuracy of fuzzy classification
is the entropy, which shows how the strength of class membership (i. e. fuzzy
memberships) in the classified image is partitioned between the classes for
each pixel. The entropy for a pixel is maximum when the pixel has equal class
memberships for all the classes. Conversely, its value is minimum, when the
pixel is entirely allocated to one class. It thus tells us the degree to which
a classification output is fuzzy or crisp. The utility of entropy may, however, be appropriate for situations in which the output of the classification
is fuzzy (Binaghi et al. 1999). Often, there exist ambiguities in the reference data, which may therefore be treated as fuzzy or uncertain (Bastin et
al. 2002). Other measures may have to be applied when the reference data
and classified image are fuzzy. Under these circumstances, measures such
as Euclidian distance, L\ distance and cross-entropy or directed divergence
may be adopted. These measures estimate the separation of two data sets
based on the relative extent or proportion of each class in the pixel (Foody
and Arora 1996). Lower the values of these measures, higher is the accuracy
of classification. To indicate the accuracy of an individual class in a fuzzy
classification, correlation coefficient (R) may be used (Maselli et al. 1996).
77
OA, PA and UA do not take into account the agreement between the data sets
(i. e. classified image and reference data) that arises due to chance alone. Thus,
these measures tend to overestimate the classification accuracy (Ma and Redmond 1995). The kappa coefficient of agreement (K) has the ability to account
for chance agreement (Rosenfeld and Fitzpatrick-Lins 1986). The proportion
of agreement by chance is the result of the misclassifications represented by the
off-diagonal elements of the error matrix. Therefore, K uses all the elements
of the error matrix, and not just the diagonal elements (as is the case with
OA). When some classes have more confusion than others, weighted kappa
may be implemented since it does not treat all the misclassifications (disagreements) equally and tends to give more weight to the confusions that are more
serious than others (Cohen 1968; Naesset 1996). To determine the accuracy
of individual classes, a conditional kappa may be computed (Rosenfeld and
Fitzpatrick-Lins 1986).
It may thus be seen that there are a number of measures that may be computed from an error matrix. Each measure may however be based on different
assumptions about the data and thus may evaluate different components of
accuracy (Lark 1995; Stehman 1999). Therefore, in general, it may be expedient to provide an error matrix with the classified image and report more
than one measure of accuracy to fully describe the quality of that classification
(Stehman 1997).
The error matrix based measures inherently assume that each pixel is associated with only one class in the classified image and only one class in the reference data. Use of these measures to assess the accuracy of fuzzy classification
may therefore under or over estimate the accuracy since a fuzzy classification
has to be degraded to adhere to this assumption. Moreover, in addition to the
classification output, often ambiguities exist in the reference data, which may
therefore be treated as fuzzy and thus other alternative measures need to be
adopted (Foody 1995a; Foody 1995b).
One of the simplest measures to assess the accuracy of fuzzy classification
is the entropy, which shows how the strength of class membership (i. e. fuzzy
memberships) in the classified image is partitioned between the classes for
each pixel. The entropy for a pixel is maximum when the pixel has equal class
memberships for all the classes. Conversely, its value is minimum, when the
pixel is entirely allocated to one class. It thus tells us the degree to which
a classification output is fuzzy or crisp. The utility of entropy may, however, be appropriate for situations in which the output of the classification
is fuzzy (Binaghi et al. 1999). Often, there exist ambiguities in the reference data, which may therefore be treated as fuzzy or uncertain (Bastin et
al. 2002). Other measures may have to be applied when the reference data
and classified image are fuzzy. Under these circumstances, measures such
as Euclidian distance, L\ distance and cross-entropy or directed divergence
may be adopted. These measures estimate the separation of two data sets
based on the relative extent or proportion of each class in the pixel (Foody
and Arora 1996). Lower the values of these measures, higher is the accuracy
of classification. To indicate the accuracy of an individual class in a fuzzy
classification, correlation coefficient (R) may be used (Maselli et al. 1996).