distribution networks and supply systems [41, 75] but also as surrogate models of
complex networks [76, 77], to simplify simulation and save computational time
The drawbacks of these data-driven approaches is that they require a large amount
of data for training (and, ideally, validation) and that they are considered, in an
operational sense, ‘black-box’ estimators, with the human actors having limited
insight of their internal structure and functionality.
3.2.4 Physical Models
Physical modelling is able to provide a reference set of (modelled) system measurements that can be used as a basis to perform faulty data identification. For instance, if
a WDN model (e.g. in EPANET, Infoworks or WaterGEMS) of the system is
available and properly calibrated, it is possible to simulate the behaviour of the
network and then extract the data on pressures, given the proper drivers (e.g., current
levels in tanks and reservoirs, demand forecast) are known. It is then possible to use
that simulation output as a comparison basis with sensor data; any deviation from
what is modelled might then be flagged as (potentially) faulty data. Deviations of the
reality from the model could indicate for instance an increase of losses due to pipe
breaks or background leakages [78] or in some cases faulty sensor behaviour
(e.g. due to service downtime).
The trade-off for utilities with setting up a physical model lies between model
reliability and computational cost. The more reliable the model is, the larger the
effort to keep models up-to-date and properly calibrated, which is a (continuous) cost
for the utility. The reduction of complexity and computational time can be curbed
with techniques such as network skeletonization [79, 80], hydraulic simplification
[81] and topological aggregation of serial pipes [82]. Another approach to reduce
computational time lies in the use of surrogate modelling techniques [83]. Even at a
higher computational cost, Model Predictive and Data Assimilation approaches are
beginning to be developed for uncertainty reduction [43, 84, 85] and as a consequence may be used to identify data anomalies.
3.2.5 Knowledge-Based Techniques
Regardless of the mathematical, statistical or modelling techniques that assist faulty
data detection, knowledge and expert-based judgment offers invaluable insight to
validation and allows the operators to reach decisions on whether flagged data is
actually faulty or not. A strategy for knowledge-based evaluation might include:
• Periodically checking the status of sensor or asset, where log files and metadata
are checked to evaluate whether a particular sensor or asset is operating well and
is well-calibrated. These checks also include periodic maintenance, such as (re-)
calibration in sensors, as these are expected to have a reduction in their reliability
and accuracy over time [86].
A Bird’s-Eye View of Data Validation in the Drinking Water Industry of the. . .
83
complex networks [76, 77], to simplify simulation and save computational time
The drawbacks of these data-driven approaches is that they require a large amount
of data for training (and, ideally, validation) and that they are considered, in an
operational sense, ‘black-box’ estimators, with the human actors having limited
insight of their internal structure and functionality.
3.2.4 Physical Models
Physical modelling is able to provide a reference set of (modelled) system measurements that can be used as a basis to perform faulty data identification. For instance, if
a WDN model (e.g. in EPANET, Infoworks or WaterGEMS) of the system is
available and properly calibrated, it is possible to simulate the behaviour of the
network and then extract the data on pressures, given the proper drivers (e.g., current
levels in tanks and reservoirs, demand forecast) are known. It is then possible to use
that simulation output as a comparison basis with sensor data; any deviation from
what is modelled might then be flagged as (potentially) faulty data. Deviations of the
reality from the model could indicate for instance an increase of losses due to pipe
breaks or background leakages [78] or in some cases faulty sensor behaviour
(e.g. due to service downtime).
The trade-off for utilities with setting up a physical model lies between model
reliability and computational cost. The more reliable the model is, the larger the
effort to keep models up-to-date and properly calibrated, which is a (continuous) cost
for the utility. The reduction of complexity and computational time can be curbed
with techniques such as network skeletonization [79, 80], hydraulic simplification
[81] and topological aggregation of serial pipes [82]. Another approach to reduce
computational time lies in the use of surrogate modelling techniques [83]. Even at a
higher computational cost, Model Predictive and Data Assimilation approaches are
beginning to be developed for uncertainty reduction [43, 84, 85] and as a consequence may be used to identify data anomalies.
3.2.5 Knowledge-Based Techniques
Regardless of the mathematical, statistical or modelling techniques that assist faulty
data detection, knowledge and expert-based judgment offers invaluable insight to
validation and allows the operators to reach decisions on whether flagged data is
actually faulty or not. A strategy for knowledge-based evaluation might include:
• Periodically checking the status of sensor or asset, where log files and metadata
are checked to evaluate whether a particular sensor or asset is operating well and
is well-calibrated. These checks also include periodic maintenance, such as (re-)
calibration in sensors, as these are expected to have a reduction in their reliability
and accuracy over time [86].
A Bird’s-Eye View of Data Validation in the Drinking Water Industry of the. . .
83