suspicious. These techniques are effective at identifying systematic errors,
e.g. due to sensor calibration faults.
Evidently, different tests are suitable for specific water system assets, e.g. there is
no point having a flat line test for a pump that gets periodically switched on or off.
Besides these simple test types, other variations are also possible depending on the
context, such as the use of slope/gradient tests or sensor-specific drift techniques;
these methods fall beyond the scope of this study.
3.2.2 Statistical Tests
Beyond simple testing for faulty data detection, statistical analyses offer a formal
framework to probabilistically detect faults, e.g. as outliers of given distributions.
The statistical tests that exist in literature are given in the following sub-sections.
Comparison of Flow Pattern Distributions (CFPD)
Time series such as inflows and consumption patterns of a WDN generally follow
patterns on a diurnal, weekly and seasonal scale. Variations in consumptions due to
season change are common (i.e. winter, summer) [42], as well as due to specific
events in a short time window [43]. In these cases, a comparison of flow pattern
distributions (CFPD) can be performed for the detection of anomalies, for instance,
based on the identification and interpretation of features in CFPD block diagrams.
Feature analysis and techniques for the automated screening of data with seasonal
statistics can be used to measure deviations for the expected value and infer the
existence of faulty data [44].
Spatial Deviation
In some cases, variables exhibit correlation in space. In these cases, a complete set of
techniques for faulty data identification can be explored with the use of
geo-statistical techniques, such as Kriging, in order to estimate deviation between
the measured value and the estimated [45]. These techniques have been mostly
applied in hydrology and fall outside of the scope of this chapter.
Extreme Value Checks
Another type of statistical approach is to perform extreme value analyses (EVA),
where the probability distributions of sensor variables are inferred from observed
80
M. Castro-Gama et al.
e.g. due to sensor calibration faults.
Evidently, different tests are suitable for specific water system assets, e.g. there is
no point having a flat line test for a pump that gets periodically switched on or off.
Besides these simple test types, other variations are also possible depending on the
context, such as the use of slope/gradient tests or sensor-specific drift techniques;
these methods fall beyond the scope of this study.
3.2.2 Statistical Tests
Beyond simple testing for faulty data detection, statistical analyses offer a formal
framework to probabilistically detect faults, e.g. as outliers of given distributions.
The statistical tests that exist in literature are given in the following sub-sections.
Comparison of Flow Pattern Distributions (CFPD)
Time series such as inflows and consumption patterns of a WDN generally follow
patterns on a diurnal, weekly and seasonal scale. Variations in consumptions due to
season change are common (i.e. winter, summer) [42], as well as due to specific
events in a short time window [43]. In these cases, a comparison of flow pattern
distributions (CFPD) can be performed for the detection of anomalies, for instance,
based on the identification and interpretation of features in CFPD block diagrams.
Feature analysis and techniques for the automated screening of data with seasonal
statistics can be used to measure deviations for the expected value and infer the
existence of faulty data [44].
Spatial Deviation
In some cases, variables exhibit correlation in space. In these cases, a complete set of
techniques for faulty data identification can be explored with the use of
geo-statistical techniques, such as Kriging, in order to estimate deviation between
the measured value and the estimated [45]. These techniques have been mostly
applied in hydrology and fall outside of the scope of this chapter.
Extreme Value Checks
Another type of statistical approach is to perform extreme value analyses (EVA),
where the probability distributions of sensor variables are inferred from observed
80
M. Castro-Gama et al.