14.3.4 Validation Limitations
The review of the 80 papers demonstrated that in many cases limited information
is being reported in the literature on the processes used to collect and analyze data
for accuracy assessment of coral reef maps. Our review of a wide variety of coral
reef mapping papers (Tables 14.3, 14.4, 14.5) showed that only 13 out of 80 papers
provided sufficient information to repeat the validation process (e.g., sample unit,
number, location, scheme), and to adequately assess and compare the accuracy
values (i.e., include the error matrix, and the accuracy measure for the overall map
and its map categories). Similar findings have also been derived from assessments
of calibration and validation approaches used in terrestrial vegetation mapping
(Trodd 1995; Foody 2002).
The reviewed literature was dominated by areas smaller than 100 km
2 , and did
not focus on larger areas, which are more common for management needs and
which typically contain a variety of reefs that need to be mapped at the same time.
In fact, a majority of the papers were based on a single small reef area, potentially
limiting the effectiveness of the application to larger areas. It is therefore suggested
that reef remote sensing studies should include testing and validation on a collection of reefs and reef types and/or on large areas of reefs.
Spot check and transect surveys were the most commonly used calibration and
validation field methods as both showed to be applicable in various environments
and circumstances (Roelfsema and Phinn 2010). Supervised classification procedures in combination with medium spatial resolution multispectral imagery was the
0
2
4
6
8
10
12
14
16
18
20
25
50
75
100
Aerial (5) High Resolution
Hyper Spectral
(4)
High Resolution
Multi Spectral
(19)
Medium
Resolution Multi
Spectral (10)
Number of mapping
categories
Areas size (km
2
)
Accuracy(%)
Sensors type (number of referenced papers)
Areas size
Accuracy
Categories
1500
2000
2500
3000
Fig. 14.3 Average size of study area mapped, mean number of categories mapped, and mean
accuracy of the output map, per remote sensing sensor type used for coral reef or sea grass habitat
mapping, derived from 38 out of 80 peer-reviewed scientific publications. The sensor types
include: aerial photography; high spatial resolution hyperspectral; high spatial resolution
multispectral; and medium spatial resolution multispectral sensors. Error bars represent standard
error for each sensor type
14 Validation
391
The review of the 80 papers demonstrated that in many cases limited information
is being reported in the literature on the processes used to collect and analyze data
for accuracy assessment of coral reef maps. Our review of a wide variety of coral
reef mapping papers (Tables 14.3, 14.4, 14.5) showed that only 13 out of 80 papers
provided sufficient information to repeat the validation process (e.g., sample unit,
number, location, scheme), and to adequately assess and compare the accuracy
values (i.e., include the error matrix, and the accuracy measure for the overall map
and its map categories). Similar findings have also been derived from assessments
of calibration and validation approaches used in terrestrial vegetation mapping
(Trodd 1995; Foody 2002).
The reviewed literature was dominated by areas smaller than 100 km
2 , and did
not focus on larger areas, which are more common for management needs and
which typically contain a variety of reefs that need to be mapped at the same time.
In fact, a majority of the papers were based on a single small reef area, potentially
limiting the effectiveness of the application to larger areas. It is therefore suggested
that reef remote sensing studies should include testing and validation on a collection of reefs and reef types and/or on large areas of reefs.
Spot check and transect surveys were the most commonly used calibration and
validation field methods as both showed to be applicable in various environments
and circumstances (Roelfsema and Phinn 2010). Supervised classification procedures in combination with medium spatial resolution multispectral imagery was the
0
2
4
6
8
10
12
14
16
18
20
25
50
75
100
Aerial (5) High Resolution
Hyper Spectral
(4)
High Resolution
Multi Spectral
(19)
Medium
Resolution Multi
Spectral (10)
Number of mapping
categories
Areas size (km
2
)
Accuracy(%)
Sensors type (number of referenced papers)
Areas size
Accuracy
Categories
1500
2000
2500
3000
Fig. 14.3 Average size of study area mapped, mean number of categories mapped, and mean
accuracy of the output map, per remote sensing sensor type used for coral reef or sea grass habitat
mapping, derived from 38 out of 80 peer-reviewed scientific publications. The sensor types
include: aerial photography; high spatial resolution hyperspectral; high spatial resolution
multispectral; and medium spatial resolution multispectral sensors. Error bars represent standard
error for each sensor type
14 Validation
391