papers, of which 21 also reported the error matrix. In general, the papers that provided
an error matrix focused on just a few maps and only one site (Lewis et al. 2003).
In terms of study repeatability, 13 papers contained a high degree of detail in
their explanation of methods, to the extent that the methods could be repeated. A
total of 35 papers contained limited or non-repeatable methods, and 32 provided
no description of methods. Some of the differences may be due to the intended
audience of the paper, who may be either managers, scientists or technicians. Of
the 32 papers focused on resource management applications, 11 provided an
overall accuracy and only 2 provided repeatable methods. Of the 48 papers
focused on a scientific/technical perspective, 28 studies reported an overall
accuracy, and 11 provided repeatable accuracy assessment methods.
Applications to define resource extent and composition of coral reefs were the
subject of 27 publications, 25 of which provided little or no information on validation. Of these 25 publications, 19 were published in biology, conservation and
management journals, suggesting that the importance was placed on the remote
sensing product and not on the process of how the map was created. This implies
that the process is undervalued. A thorough understanding of the methods and
validation process of the mapped products will result in more reliable and better
informed management decisions. Therefore, when requesting remote sensing
products and studies, the users of mapping products should require a report that
includes both an explanation of the mapping approach used and a thorough
description of the validation process, including sample design, sample unit, sample
number, sample distribution, error matrix and accuracy measures.
Amongst the various applications, very limited descriptions of validation data
collection and accuracy assessment were provided in papers that focused on use of
benthic field data and acoustic image data, as well as those using radiative transfer
models with analytic or semi-analytic solutions. These papers focused mainly on
retrieval of bathymetric information or substrate reflectance. As a result, detailed
accuracy information was provided mostly for the validation of bathymetry, but was
limited for the validation of benthic habitat maps. Additionally, error propagation
should not be neglected and needs attention when developing systems that merge
different sensor types (Aitken et al. 2010). The fusion algorithms that merge different
data types to create new information products can improve overall product accuracy,
but will also integrate systematic errors introduced by the individual sensors.
A synthesis of all 80 papers was conducted to reveal how the reported validation
characteristics varied by sensor type (Fig. 14.3). Results show that medium spatial
resolution multispectral sensors (e.g., Landsat Thematic Mapper) were mostly used
to map large areas (ranging from 16 to 21,377 km
2 ; average of 2,362 km
2 ) with a
limited number of categories (typically around 6) and an average accuracy of 65 %.
Smaller areas were mapped using high spatial resolution hyperspectral (average
area of 46 km
2 with 80 % accuracy) and multispectral (average area of 44 km
2 with
72 % accuracy). However, the average number of categories mapped was higher for
the medium spatial resolution multispectral sensor (13) than for the hyperspectral
sensors (8). Aerial photography was used for a variety of area sizes, averaging an
area of 71 km
2 , 73 % accuracy and 14 categories.
390
C. M. Roelfsema and S. R. Phinn
an error matrix focused on just a few maps and only one site (Lewis et al. 2003).
In terms of study repeatability, 13 papers contained a high degree of detail in
their explanation of methods, to the extent that the methods could be repeated. A
total of 35 papers contained limited or non-repeatable methods, and 32 provided
no description of methods. Some of the differences may be due to the intended
audience of the paper, who may be either managers, scientists or technicians. Of
the 32 papers focused on resource management applications, 11 provided an
overall accuracy and only 2 provided repeatable methods. Of the 48 papers
focused on a scientific/technical perspective, 28 studies reported an overall
accuracy, and 11 provided repeatable accuracy assessment methods.
Applications to define resource extent and composition of coral reefs were the
subject of 27 publications, 25 of which provided little or no information on validation. Of these 25 publications, 19 were published in biology, conservation and
management journals, suggesting that the importance was placed on the remote
sensing product and not on the process of how the map was created. This implies
that the process is undervalued. A thorough understanding of the methods and
validation process of the mapped products will result in more reliable and better
informed management decisions. Therefore, when requesting remote sensing
products and studies, the users of mapping products should require a report that
includes both an explanation of the mapping approach used and a thorough
description of the validation process, including sample design, sample unit, sample
number, sample distribution, error matrix and accuracy measures.
Amongst the various applications, very limited descriptions of validation data
collection and accuracy assessment were provided in papers that focused on use of
benthic field data and acoustic image data, as well as those using radiative transfer
models with analytic or semi-analytic solutions. These papers focused mainly on
retrieval of bathymetric information or substrate reflectance. As a result, detailed
accuracy information was provided mostly for the validation of bathymetry, but was
limited for the validation of benthic habitat maps. Additionally, error propagation
should not be neglected and needs attention when developing systems that merge
different sensor types (Aitken et al. 2010). The fusion algorithms that merge different
data types to create new information products can improve overall product accuracy,
but will also integrate systematic errors introduced by the individual sensors.
A synthesis of all 80 papers was conducted to reveal how the reported validation
characteristics varied by sensor type (Fig. 14.3). Results show that medium spatial
resolution multispectral sensors (e.g., Landsat Thematic Mapper) were mostly used
to map large areas (ranging from 16 to 21,377 km
2 ; average of 2,362 km
2 ) with a
limited number of categories (typically around 6) and an average accuracy of 65 %.
Smaller areas were mapped using high spatial resolution hyperspectral (average
area of 46 km
2 with 80 % accuracy) and multispectral (average area of 44 km
2 with
72 % accuracy). However, the average number of categories mapped was higher for
the medium spatial resolution multispectral sensor (13) than for the hyperspectral
sensors (8). Aerial photography was used for a variety of area sizes, averaging an
area of 71 km
2 , 73 % accuracy and 14 categories.
390
C. M. Roelfsema and S. R. Phinn